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ABSTRACT 


i 


The  problem  of  computing  an  optimal  search  plan  for  a moving  target  is 
addressed  wh^n  the  searcher  can  distribute  his  effort  as  finely  as  he  wishes. 

'~->The  majority  of  the  report  describes  numerical  techniques  which  have  been 
developed  to  compute  optimal  search  plans  for  the  very  broad  class  of  problems 
in  which  the  target's  motion  can  be  modeled  by  a discrete-time  stochastic  process 
and  the  detection  function  is  exponential. 


I A very  efficient  algorithm  is  given  to  find  optimal  search  plans  when  the 

{ target's  motion  is  modeled  by  a mixture  of  discrete  time  and  space  Markov 

processes.  A second  algorithm  is  presented  to  solve  the  variation  of  this 
problem  that  one  encounters  when  the  search  effort  at  each  time  period  is 
restricted  to  be  uniform  over  an  arbitrary  rectangular  region.  The  latter  is 
intended  to  approximate  the  problem  of  choosing  a sequence  of  sonobuoy  fields 
to  maximize  the  probability  of  detecting  a submarine.  Examples  show  that  one 
can  often  find  rectangular  plans  that  are  almost  as  effective  as  the  optimal  plan. 
In  addition  to  the  above,  an  algorithm  is  presented  to  find  optimal  plans  for 
arbitrary  discrete  time  target  motion  processes  which  can  be  modeled  by  Monte 
Carlo  simulation.  All  the  algorithms  have  been  programmed  in  FORTRAN  and 
run  on  a Prime  400  minicomputer.  Examples  of  optimal  plans  calculated  by 
these  algorithms  are  presented. 

f— 

All  the  algorithms  are  based  on  a very  general  necessary  and  sufficient 
condition  for  optimal  search  for  a moving  target  which  is  proved  in  this  report. 
For  discrete  time  and  an  exponential  detection  function  this  condition  becomes: 


For  a search  plan  for  a moving  target  to  be  optimal,  it  is  necessary 
and  sufficient  that  at  each  time  t it  assign  an  allocation  which  is 
optimal  for  the  stationary  target  problem  which  one  obtains  at 
time  t by  conditioning  on  failure  to  detect  after  t as  well  as  before 
t under  the  plan. 
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Algorithms  are  also  offered 
involving  non -exponential  detection 


for  minimizing  mean  time  to  detect,  for  searches 
functions,  and  for  survivor  search  problems. 
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ABSTRACT 
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The  problem  of  computing  an  optimal  search  plan  for  & moving  target  is 
addressed  when  the  searcher  can  distribute  his  effort  as  finely  as  he  wishes. 

The  majority  of  the  report  describes  numerical  techniques  which  have  been 
developed  to  compute  optimal  search  plans  for  the  very  broad  clasB  of  problems 
in  which  the  target'  n motion  can  be  modeled  by  a discrete -time  stochastic  process 
and  the  detection  function  is  exponential. 

A very  efficient  algorithm  is  given  to  find  optimal  search  plans  when  the 
target's  motion  is  modeled  by  a mixture  of  discrete  time  and  space  Markov 
processes.  A second  algorithm  is  presented  to  solve  the  variation  of  this 
problem  that  one  encounters  when  the  search  effort  at  each  time  period  is 
restricted  to  be  uniform  over  an  arbitrary  rectangular  region.  The  latter  is 
intended  to  approximate  the  problem  of  choosing  a sequence  of  aonobuoy  fields 
to  maximize  the  probability  of  detecting  a submarine.  Examples  show  that  one 
can  often  find  rectangular  plans  that  are  almost  as  effective  as  the  optimal  plan. 

In  addition  to  the  above,  an  algorithm  is  presented  to  find  optimal  plans  for 
arbitrary  discrete  time  target  motion  processes  which  can  be  modeled  by  Monte 
Carlo  simulation.  All  the  algorithms  have  been  programmed  in  FORTRAN  and 
run  on  a Prime  400  minicomputer.  Examples  of  optimal  plans  calculated  by 
these  algorithms  are  presented. 

All  the  algorithms  are  based  on  a very  general  necessary  and  sufficient 
condition  for  optimal  search  for  a moving  target  which  is  proved  in  this  report. 

For  discrete  time  and  an  exponential  detection  function  this  condition  becomes: 

For  a search  plan  for  a moving  target  to  be  optimal,  it  is  necessary 
and  sufficient  that  at  each  time  t it  assign  an  allocation  which  is 
optimal  for  the  stationary  target  problem  which  one  obtains  at 
time  t by  conditioning  on  failure  to  detect  after  t as  well  as  before 
i under  the  plan. 

Algorithms  are  also  offered  for  minimizing  mean  time  to  detect,  for  searches 
Involving  non -exponential  detection  functions,  and  for  survivor  search  problems. 
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PREFACE 


This  is  a report  by  Daniel  H.  Wagner,  Associates  to  the  Naval  Analysis  Divi- 
sion of  the  Office  of  Naval  Research  (Code  431)  on  work  performed  under  ONR 
Contract  No.  N00014-76-C-0696.  This  report  summarizes  the  remarkable  pro- 
gress made  since  the  fall  of  1976  in  the  computation  of  optimal  search  plans  for 
moving  targets. 

We  would  like  to  express  our  appreciation  for  the  excellent  cooperation  and 
support  that  has  been  given  to  this  work  by  Mr.  J.  Randolph  Simpson  and  CDR 
Ronald  James  of  ONR  (Code  431). 

In  reference  [a],  S.  S.  Brown  made  the  key  observation  that  the  necessary 
and  sufficient  conditions  for  optimal  search  for  a target  moving  in  discrete  space 
and  time  with  an  exponential  detection  function  could  be  stated  in  terms  of  solving 
a sequence  of  stationary  target  problems.  In  each  of  these  stationary  searches 
the  target  position  distributions  are  conditioned  on  non-success  of  future  as  well 
as  past  search  under  the  plan.  This  observation  allowed  him  to  produce  a very 
efficient  algorithm  for  finding  optimal  search  plans  for  targets  whose  motion 
can  be  modeled  by  a mixture  of  discrete  time  and  space  Markov  processes. 

Chapter  n of  this  report  is  based  on  reference  [b],  which  is  a revised  version 
of  Brown's  work  reported  in  reference  [a].  The  work  on  rectangular  search 
plans  discussed  in  Chapter  III  is  based  on  work  by  R.  P.  Bueml  reported  in 
references  (ts)  and  (d).  This  work  makes  use  of  many  of  the  ideas  of  Brown 
discussed  in  Chapter  n. 

In  reference  |e),  L.  D.  Stone  showed  that  the  condition  found  by  Brown  holds 
when  the  search  space  is  continuous  as  well  as  discrete.  This  is  the  basic  con- 
dition given  in  Chapter  I.  In  addition  he  found  a generalization  of  the  condition 
for  non-exponential  detection  functions  and  continuous  ttrne,  again  Involving 
conditioning  upon  future  as  well  as  past  search.  This  allowed  a unified  state- 
ment of  the  necessary  and  sufficient  conditions  for  a very  broad  class  of  moving 
target  problems  for  any  combination  of  discrete  or  continuous  space  and  discrete 
or  continuous  time.  This  statement  is  given  in  Chapter  VI. 

Using  this  condition,  Stone  (reference  [f])  devised  an  algorithm  to  find  optimal 
search  plans  when  the  detection  function  is  exponential  and  the  target  motion  can 
be  modeled  by  an  arbitrary  discrete-time  Monte  Carlo  simulation.  This  algorithm 
was  implemented  by  C.  R.  Hopkins  (see  reference  (gj)  on  a Prime  400,  Hopkins' 
program  was  used  to  produce  the  examples  in  Chapter  IV. 
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SUMMARY 

This  report  summarizes  the  recent  progress  that  has  been  made  In  finding  optimal 
se.  ~ch  plans  for  moving  targets. 

In  the  recent  past,  our  ability  to  compute  optimal  search  plans  for  detecting 
moving  targets  was  limited  to  special  situations  such  as  those  involving  target  motion 
that  is  conditionally  deterministic  with  a factorable  Jacobian  or  that  Is  two-celled 
Markovian  (see  Chapters  8 and  9 of  reference  (hi).  Although  the  target  motion  In  some 
Naval  applications  can  be  modeled  by  the  factorable  conditionally  deterministic 
motion,  most  cannot,  and  Markovian  motion  in  any  small  number  of  cells  is  not  a 
realistic  motion  model  for  most  operational  situations. 

Since  the  fall  of  1976  we  have  developed  optimization  techniques  which  allow  one 
to  find  optimal  allocations  for  multlscenarlo  Markovian  motions  in  discrete  space  and 
time.  These  multiscenario  Markovian  motions  are  general  enough  to  model  a 
wide  range  of  target  motions.  In  addition,  we  have  developed  an  optimization  program 
for  arbitrary  discrete  time  target  motion  processes  that  can  be  represented  by  a 
Monte  Carlo  simulation.  This  latter  program  can  be  coupled  with  computer  assisted 
search  programs  such  as  COMPASS,  MEDSEARCH,  or  TARDIST  which  are  now  being  used  on  a 
regular  basis  to  provide  search  planning  advice  for  actual  submarine  searches  in  the 
Mediterranean  and  Atlantic.  The  program  designed  for  arbitrary  discrete  time  target 
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motions  is  very  general;  however,  it  pays  the  price  of  being  slower  to  run  than  the 
program  designed  only  for  multiscenario  Markovian  motions.  Programs  suoh  as 
COMSUBPAC's  ASP  and  SASP  could  be  modified  to  incorporate  the  basic  optimisation 
algorithm  described  in  the  third  section  below. 

The  class  of  search  problems  that  can  be  solved  by  these  programs  is  very  broad 
but  there  are  two  important  restrictions.  First  the  detection  function  must  be 
exponential,  and  second  the  searcher  is  assumed  to  have  the  ability  to  spread  his 
effort  over  large  areas  in  a single  time  period.  Thus,  the  allocations  obtained  from 
these  programs  would  not  be  appropriate  for  a submarine  searching  for  a submarine 
but  would  be  more  reasonable  for  a VP  aircraft  ~:chlng  for  a submarine.  Even 
in  the  latter  case,  the  optimal  allocations  of  effort  are  probably  too  complicated  for 
operational  use,  although  they  can  suggest  the  general  nature  of  the  plan  to  be  followed. 
In  Chapter  III  of  this  report  we  address  this  problem  by  producing  an  algorithm  that 
finds  near-optimal  plans  which  are  restricted  to  allocate  their  effort  uniformly  over 
a single  rectangle  during  each  time  period.  In  fact,  the  methods  described  in  Chapter 
III  have  been  adapted  by  COMPATWINGSPAC  for  use  in  VP  search  planning. 

In  the  remainder  of  this  summary,  we  describe  the  basic  search  problem  and 
outline  the  results  obtained  for  this  problem. 

Basic  Problem 

The  target’s  motion  is  represented  by  a stochastic  process 

X = {Xt;  t - 1 T} 

where  Xt  is  a random  variable  which  gives  the  target's  position  at  time  t.  The  distri- 
bution of  Xt  is  simply  the  probability  distribution  of  the  target's  location  at  time  t. 
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In  this  report,  two  basic  types  of  target  motion  processes  are  considered. 

The  first  type,  considered  in  Chapters  II  and  111,1s  the  multi -scenario  Markovian 
motion  In  discrete  space  and  time.  The  second  type  is  an  arbitrary  discrete  time 
target  motion  process  which  is  represented  by  a large  but  finite  number  of  sample 
paths  from  a Monte  Carlo  simulation  of  the  process.  This  type  Is  considered  primarily 
in  Chapter  IV.  This  second  type  of  motion  is  very  general  in  character;  It  need  not  be 
Markovian  and  can  take  place  In  a discrete  or  continuous  space. 

For  most  of  the  report,  we  assume  that  a grid  has  been  established  on  the  search 
space  which  is  two-dimensional  and  that  search  effort  must  be  applied  uniformly  within 
a given  cell  of  the  grid,  although  effort  may  vary  from  cell  to  cell.  Let  J denote  the 
set  of  cells  in  the  grid  and 

Pt0)  * Pr{Xt  e cell  J}  for  je  J,  t = 1,  ....  T, 

A(J)  = area  of  cell  for  j€  J. 

Although  we  Impose  this  grid  structure,  the  -_rget  motion  may  take  place  In  either 
discrete  or  continuous  space.  We  assume  that  the  detection  function  is  exponential,  i.e. , 


Is  the  probability  of  detecting  the  target  with  z amount  of  effort  placed  In  the  j**1  cell 
given  the  target  is  in  that  cell.  Here,  W(j)  is  the  sweep  width  or  effectiveness 
parameter  for  search  in  cell  J. 

A search  plan  is  described  by  a nonnegative  function  ip  of  space  and  time  such 

that 


v 


t)  = effort  placed  In  cell  j at  time  t for  je  J,  t = 1,  . . . , T. 
The  amount  of  effort  available  is  constrained  so  that 

m(t)  = the  effort  availabe  at  time  t. 

We  define  *(m)  to  be  the  class  of  search  plans  ip  such  that 


2 = m(t)  for  t = 1,  . . . , T. 

jeJ 

Let  E denote  expectation  taken  over  the  sample  paths  of  the  process  X.  For 
any  search  plan  ip  the  probability,  P^[ip],  of  detecting  the  target  by  time  T is  given  by 

PTW  = E |jL  -exp^-  2 W(Xt)^(Xt,t)/A(Xt) 

The  basic  search  problem  considered  in  this  report  is  to  find  a plan  ip*  e ^(m)  that 
maximizes  the  probability  of  detecting  the  target  by  time  T over  all  plans  in  the 
class  ’J'(m).  Mathematically  stated,  we  seek  ip*  e ^(m)  such  that 

PT[ip*]  = max{P T[ip]:>P  e *(m)}. 

Such  a plan  is  called  T -optimal  within  Sfr(m). 

Basic  Necessary  and  Sufficient  Conditions 

The  main  optimization  algorithms  discussed  in  this  report  are  based  on  the 
following  necessary  and  sufficient  condition  for  a search  plan  to  be  T-optimal  within 
’fc(m).  The  form  of  the  condition  given  here  applies  to  any  discrete  time  target  motion 
process  and  exponential  detection  function.  For  any  search  plan  ip,  let 

vi 
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g^O.t) 

Note  that  g^(j,t)  Is  conditioned  on  failure  to  detect  both  before  and  after  time  t. 

BASIC  CONDITION:  In  order  that  tf>*€  4-(m)  be  T-optimal.  it  is 
necessary  and  sufficient  that  the  allocation  <p*0 . t)  maximize  the 
probability  of  detecting  a stationary  target  with  distribution  g ^ (•  f t) 
and  effort  m(t)  for  t = 1.  ....  T. 

Description  of  the  Basic  Algorithm 

The  algorithms  for  finding  T-optimal  plans  operate  in  the  following  manner. 
Beginning  at  time  t = 1,  they  find  an  optimal  allocation  of  m(l)  effort  for  the  initial 
target  distribution.  For  times  t = 2,  ....  T,  they  calculate  the  posterior  target 
location  distribution  at  time  t given  failure  to  detect  at  all  previous  times  and 
allocate  m(t)  effort  in  a manner  that  would  be  optimal  for  that  stationary  target 
problem.  The  plan  resulting  from  the  first  pass  is  the  myopic  or  incrementally 
optimal  plan. 

Subsequent  passes  operate  In  the  following  manner  for  t = 1,  ....  T.  The 
algorithms  compute  g (• , t)  where  ip  represents  the  most  recent  allocation  obtained. 
They  then  reallocate  the  effort  at  time  t to  be  optimal  for  the  distribution  g^(.  ,t)  and 
change  ip  to  reflect  that  reallocation. 

By  performing  enough  passes,  one  can  come  as  close  to  the  optimal  search  plan 
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target is  in  celt  j 
at  time  t 


failure  to  detect  at  all  times 
other  than  t using  plan  ip 


Multi -Scenario  Markovian  Motion 


Chapter  11  considers  the  problem  of  finding  T-opttmal  plans  when  the  target 

motion  is  described  by  a mixture  of  discrete  time  and  spaoe  Markov  chains.  Each 

ohain  represents  a possible  target  motion  scenario.  Suppose  there  are  N scenarios. 

Then  for  n * 1,  ....  N,  the  user  may  specify  a Markov  chain  X°  = { 3^;  t ■ 1,  ....  T} 

th 

to  model  the  motion  in  that  soenario.  The  n rcenarlo  is  given  weight  aQ  where 

2N  , a = 1.  If  p°(l)  Is  the  probability  that  the  target  is  located  in  cell  J at  time  t 
n=l  n t 

under  scenario  n,  then  the  overall  probability  of  the  target  being  located  in  cell  ] at 
time  t is 

N 

- z c,  P>- 
n-1 

The  target  motion  assumptions  are  translated  into  transition  functions  for  the 
Markov  chains.  These  functions  may  be  time  dependent.  One  can  specify  the  target's 
Initial  distribution.  1.  e. , his  distribution  at  time  1,  and  constrain  the  target  to  have 
any  desired  distributions  at  any  subset  of  the  remaining  times  { 2, . . . , T}  . This 
feature  may  be  used  to  model  geographic  constraints,  l.e. , If  the  target  distribution 
must  squeeze  down  to  funnel  through  a strait,  one  can  model  this  by  using  the 
constraints. 

As  well  as  describing  the  algorithm  used  to  compute  the  optimal  allocation, 
Chapter  n gives  examples  of  optimal  plans  calculated  by  the  program  that  Implements 
the  algorithm  on  a Prime  400  mini -computer. 

One  of  the  examples  considered  is  a constrained  Markovian  fan.  The  target's 
initial  distribution  is  circular  normal  with  a standard  deviation  of  6 miles.  It  Is 


centered  In  the  middle  of  cell  (0.  0)  and  truncated  by  a 15  mile  by  15  mile  Bquare 
whose  center  coincides  with  the  center  of  the  distribution.  The  orientation  of  the 
square  matches  that  of  the  grid.  The  target's  motion  is  single  scenario  Markovian 
with  the  target  choosing  its  course  uniformly  from  150°  to  210°  and  Its  speed  between 
6 and  12  kts  with  a best  speed  of  9 kts,  which  is  given  a weight  of  1. 5;  the  speed 
distribution  is  truncated  triangular  as  shown  in  Figure  S-l.  Independent  draws  are 
made  from  these  distributions  each  hour.  At  hour  8 the  target  is  constrained  to  faave 
a truncated  normal  distribution  which  is  equal  to  the  initial  one  translated  60  miles 
south.  If  the  constraint  at  hour  8 were  removed,  this  type  of  problem  would  be 
similar  to  that  faced  by  a VP  aircraft  trying  to  redetect  a submarine  on  which  contact 
has  been  recently  lost.  Example  1 of  Chapter  II  presents  such  a case. 

Table  S-l  shows  how  the  target  density  changes  over  the  first  four  hours  in  the 
absence  of  search.  The  target  moves  southward  and  diffuses  from  its  original 
distribution.  During  the  last  four  hours  the  target  density  is  a mirror  image  of  the 
first  four  hours.  It  still  moves  southward  but  converges  back  to  a truncated  normal 
distribution. 

Ninety  units  of  search  effort  are  available  each  hour.  Table  S-2  compares  the 
detection  probabilities  achieved  by  the  myopic  plan  at  the  end  of  each  hour  to  those 
achieved  by  plans  which  are  optimal  for  2,  4,  6,  and  8 hours.  Observe  that  no  optimal 
plan  significantly  outperforms  the  myopic  plan.  The  myopic  plan  remains  near  optimal 
for  many  searches  although  Example  3 In  Chapter  II  and  Example  3 in  Chapter  IV 
give  situations  in  which  the  optimal  plan  is  significantly  better  than  the  myopic. 

Table  S-3  shows  the  optimal  plan  for  8 hours.  Again  only  the  tables  for  the  first  four 


FIGURE  S-l 

TRIANGULAR  DISTRIBUTION  ON  TARGET  SPEED 


Note:  A weight  of  1.  5 means  that  the  probability  density  at  the  best  speed  Is  1. 5 

times  that  at  the  minimum  and  maximum  speeds  which  have  equal  probability 
densities. 
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TARGET  DENSITY  FOR  THE  CONSTRAINED  MARKOVIAN  FAH 

Note:  (1)  Entries  represent  thousandths  of  a target. 

- (2)  Cells  are  3 ml  x 3 mi. 
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TABLE  8-1  (continued) 
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Hours  5-8  are  a mirror  reflection  of  hours  1-4. 
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DETECTION  PROBABILITIES  FOR  THE  CONSTRAINED  MARKOVIAN  FAN 
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TABLE  S-3 


SEARCH  PLANS  FOR  EXAMPLE  2 
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Notes:  (1)  Entries  represent  thousandths  of  a unit  of  search  density  (effort/ml  ) 
(2)  Cells  are  3 ml  x 3 ml. 
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Hours  5-8  are  a mirror  reflection  of  hours  1-4. 
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hours  are  displayed  since  the  last  four  hours  are  a mirror  image  of  the  first  four 
hours.  At  hour  1 the  optimal  plan  uses  most  of  its  effort  to  surround  the  target  rather 
than  searching  the  highest  probability  cells  as  the  myopic  plan  does  (see  the  end  of 
Table  S-3).  By  hour  4,  however,  the  optimal  plan  concentrates  its  effort  in  the 
center  of  the  target  distribution . 

The  conclusion  that  myopic  plans  are  almost  as  good  as  optimal  plans  in  many 
situations  has  important  operational  implications.  In  general,  the  optimal  plan  for 
time  T + T'  is  not  a continuation  of  the  optimal  plan  for  time  T.  Thus  if  one  searches 
optimally  for  four  hours,  one  cannot  extend  this  to  an  optimal  searoh  for  eight  hours. 
Optimal  plans  are  time-horizon  dependent.  However,  the  myopic  plan,  which  simply 
maximizes  the  increase  of  detection  probability  at  eaoh  time  Increment,  can  always 
be  continued.  In  addition,  the  optimal  plans  typically  pay  a penalty  in  probability  of 
detection  at  the  early  hours  in  order  to  maximize  that  probability  at  time  T.  With  a 
myopic  plan  no  such  penalty  is  incurred  and  often  the  resulting  probability  of  detection 
at  the  end  of  any  amount  of  time  is  close  to  optimal.  Thus  when  the  myopic  plan  is 
close  to  optimal,  as  it  is  in  many  of  the  examples  calculated  for  this  report,  the 
myopic  plan  is  a good  one  to  use  for  operational  purposes. 

Even  the  myopic  plans  may  be  difficult  to  implement  operationally  because  they 
often  call  for  fine  distributions  of  effort  over  large  areas.  Chapter  IQ  considers 
search  plans  for  multi -scenario  target  motion  when  at  each  time  period  one  is 
restricted  to  allocating  the  available  effort  uniformly  over  a rectangle.  The  size, 
location,  and  orientation  of  the  rectangle  is  chosen  by  the  search  planner.  This 
restriction  is  Intended  to  correspond  to  allocating  sonobuoys  uniformly  in  an  area. 
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An  algorithm  was  developed  to  find  good  search  rectangles,  and  It  has  been  pro- 
grammed on  a Prime  400  mini-computer.  Chapter  III  presents  some  examples  of 
rectangle  plans  found  by  this  algorithm.  In  particular,  plans  were  computed  for  the 
above  example.  Table  S-4  shows  the  rectangles  obtained  for  hours  1 through  4. 

Table  S-5  compares  the  probability  of  detection  obtained  from  the  rectangle  plan  to 
that  obtained  by  the  myopic  plan  and  the  optimal  plan  for  eight  hours  obtained  above. 
Observe  that  the  probability  of  detection  resulting  from  the  rectangular  plan  is 
remarkably  close  to  both  the  optimal  plan  and  the  myopic  plan. 

Arbitrary  Discrete  Time  Target  Motion 

In  Chapter  IV  we  outline  a method  for  finding  T -optimal  search  plans  for  any 
discrete  time  target  motion  prooess  that  can  be  represented  by  a Monte  Carlo  simulation. 
Observe  that  this  class  of  target  motions  is  very  broad.  For  example,  the  target  Is 
not  restricted  to  move  among  a set  of  cells  as  is  usually  the  case  when  one  deals  with 
Markov  chain  models  of  target  motion.  The  motion  process  need  not  be  Markovian 
or  even  a mixture  of  Markov  prooesses.  The  process  can  be  Gaussian,  a constrained 
diffusion,  or  a random  movement  through  a network.  The  program  developed  to 
perform  the  optimization  for  this  class  of  motion  processes  can  be  coupled  with  the 
Monte  Carlo  computer  assisted  search  programs  such  as  COMPASS  or  MEDSEARCH  to 
find  optimal  allocations  of  search  effort  over  any  time  interval  of  Interest  for  any 
target  motion  processes  produced  by  these  programs.  In  fact,  the  COMPA8S  programs 
were  used  to  generate  the  target  motion  process  for  the  examples  in  Chapter  IV. 

The  accuracy  of  the  optimization  will  be  related  to  the  aocuracy  with  which  the 
Monte  Carlo  simulation  represents  the  target  motion  process.  The  algorithm  considers 
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only  search  allocations  that,  during  one  time  period,  are  constant  within  the  oells  of 
a grid  oboaen  by  the  user.  Observe  that  this  is  only  a restriction  on  the  olass  of 
allocations  which  are  considered  and  is  not  equivalent  to  assuming  that  the  target 
motion  takes  place  in  discrete  space.  The  algorithm  described  in  Chapter  IV  is  more 
general  than  the  one  In  Chapter  Q for  mixtures  of  Markovian  processes.  However,  it 
does  in  effect  optimize  over  a sample  of  the  target  motion  process  whereas  the 
algorithm  in  Chapter  n treats  the  target  motion  exactly.  In  addition  computing  time 
for  the  more  general  algorithm  is  longer  than  that  for  a mixture  of  Markov  chains. 

So,  in  the  case  where  a mixture  of  disorete  time  and  space  Markov  chains  provides  a 
good  representation  of  the  target's  motion,  the  algorithm  in  Chapter  n will  be  more 
accurate  and  faster. 

Three  examples  of  optimal  allocations  obtained  by  the  general  disorete  time 
optimizer  are  given  in  Chapter  IV.  One  of  these  examples  Involves  a fleeing  target 
problem  similar  to  the  one  described  in  reference  li  ],  page  17.  The  target's  Initial 
distribution  is  circular  normal  with  standard  deviation  20  miles  in  any  direction.  The 
target  is  assumed  to  be  traveling  at  10  kts  along  a constant  course  which  is  chosen 
from  a uniform  distribution  on  (0°,  360°J.  Reference  [i  ] was  able  to  compute  the  target 
location  distribution  for  this  problem  as  a function  of  time  when  no  searoh  effort  is 
applied.  However,  optimal  plans  for  this  problem  were  not  found.  Tables  IV-1  and 
IV -2  show  the  myopic  and  optimal  plans  for  seven  hours  of  search  when  the  searcher 
begins  his  search  at  hour  4.  The  optimal  allocation  concentrates  its  effort  much  more 
heavily  in  the  center  of  distribution  at  hour  4 than  the  myopic  plan.  Comparing  the 
optima!  plan  to  the  myopic  for  the  remaining  time  intervals,  one  sees  that  the  optimal 
plan  generally  concentrates  its  effort  more  than  the  myopic  plan.  Also  it  appears  that 
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at  hour  5 the  optimal  plan  chooses  to  concentrate  Its  search  more  heavily  on  the 
eastern  side  of  the  distribution  and  then  to  make  up  for  that  by  concentrating  Its  effort 
on  the  western  side  at  hours  6 and  7. 

Although  the  myopic  plan  and  optimal  plan  are  qualitatively  different,  the  resulting 
probabilities  of  detection  as  shown  in  Table  IV -3  are  strikingly  close.  This  feature 
held  true  for  several  variations  on  this  fleeing  target  problem-,  in  particular,  when  the 
initial  distribution  was  uniform  over  a 60  mile  by  60  mile  rectangle  and  when  the 
target's  speed  was  allowed  to  be  drawn  from  a uniform  distribution  over  6-14  kts. 

Example  3 of  Chapter  IV  shows  a significant  Improvement  in  the  optimal  plan  over 
the  myopic  one.  This  example  involves  multiscenario  target  motion  and  regions  of 
varying  detection  capability.  The  detection  probabilities  for  the  optimal  and  myopic 
plans  are  shown  in  Table  IV-9.  The  optimal  has  19%  better  probability  of  detection 
than  the  myopic  (i.  e. , .58  versus  . 48  for  four  hours  of  search). 

Algorithms  for  Related  Problems 

The  method  of  designing  the  algorithms  in  Chapters  II  and  IV  can  be  used  with 
some  variations  on  a wide  range  of  optimal  search  problems.  In  Chapter  V we  outline 
algorithms  which  can  be  used  to  find  optimal  plans  when  the  detection  function  is  not 
exponential,  to  find  plans  which  minimize  mean  time  to  complete  a search,  and  to 
find  optimal  allocations  of  effort  when  there  is  a constraint  on  total  effort  available 
but  not  on  the  rate  at  which  effort  may  be  applied.  We  also  give  an  algorithm  for 
maximizing  the  probability  of  finding  a target  alive  when  the  target  has  a stochastic 


lifetime  which  varies  with  location. 


These  proposed  algorithms  have  not  been  programmed  or  tested.  They  should 
be  thought  of  as  first  approaches  to  solving  these  problems,  and  are  included 
simply  to  illustrate  the  methods  which  can  be  used  to  attack  these  problems. 

General  Necessary  and  Sufficient  Conditions 

Chapter  VI  finds  necessary  and  sufficient  conditions  for  a search  plan  to  maximise 
the  probability  of  detecting  a moving  target  by  time  T under  constraints  on  the  rate 
at  which  search  effort  may  be  applied.  These  conditions  apply  to  a broad  class  of 
moving  target  problems  in  continuous  or  discrete  time  and  in  a continuous  or  discrete 
space.  Many  previous  results  concerning  necessary  and  sufficient  conditions  for 
moving  target  problems  are  special  cases  of  these  results.  In  particular  the  basic 
condition  stated  in  the  beginning  of  this  summary  is  a special  case  of  the  results  in 
Chapter  VI. 
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NUMERICAL  OPTIMIZATION  Or  SEARCH  FOR  A MOVING  TARGET 


cjlapt_er_l 

INTRODUCTION 


Since  the  fall  of  1976,  remarkable  progress  has  been  made  in  our  ability  to 
compute  optimal  search  plans  for  moving  targets.  This  report  summarizes  that 
progress . 

Prior  to  the  work  reported  here  there  were  few  situations  in  which  one  could 
find  optimal  searcn  plans  for  moving  targets . These  situations  typically  involved 
two-celled  Markovian  motion  or  special  types  of  conditionally  deterministic  target 
motion;  see  Chapters  8 and  9 of  reference  (hj.  In  this  report  we  discuss  teohniques 
which  allow  us  to  find  optimal  search  plans  whenever  the  target  motion  Is  modeled 
by  a discrete-time  stochastic  process  in  either  continuous  or  discrete  space  and 
a fixed  amount  of  effort  must  be  applied  at  each  time  period.  It  is  assumed  that  at 
each  time  period  the  search  effort  may  be  distributed  as  finely  as  desired  over 
the  search  space.  The  optimization  techniques  presented  in  this  report  are  primarily 
limited  to  exponential  detection  functions. 

Observe  that  the  class  of  allowable  target  motions  Is  essentially  unrestricted 
except  that  a discrete-time  motion  model  must  be  used.  In  addition  to  covering 
Markovian  motion,  the  class  encompasses  any  of  the  Monte  Carlo  target  motion 
processes  used  by  the  Navy's  computer-assisted  search  systems  such  as  COMPASS* 


MEDSEARCH,  or  TARDIST,  see  references  [j]  and  [k]. 

The  assumption  that  the  searcher  can  distribute  his  effort  as  finely  as  he  wishes 
is  an  important  restriction.  It  effectively  prevents  us  from  using  these  results  to 
plan  the  search  of  one  sul  marine  for  another.  However,  the  results  do  apply  to 
searches  where  the  searcher  can  move  much  faster  than  the  target,  e.g. , patrol 
aircraft  searching  for  a submarine  with  sonobuoys  or  aircraft  searching  visually 
for  a person  adrift  In  a life  raft. 

The  problem  of  optimal  search  for  a submarine  by  patrol  aircraft  motivated 
much  of  the  work  presented  in  this  report.  Using  the  algorithms  presented  here, 
one  can  provide  good  advice  on  search  allocation  for  a wide  range  of  problems  Involving 
VP  aircraft  searching  for  submarines  with  sonobuoys.  In  fact  the  techniques  in  Chapter  HI 
have  already  been  adapted  by  COMPATW1NGSPAC  for  use  in  search  planning. 

Our  approach  to  solving  the  optimal  search  problem  considered  here  is  to  find 
a set  of  necessary  and  sufficient  conditions  for  an  optimal  plan.  We  then  design  an 
algorithm  to  find  plans  which  satisfy  these  conditions  and  are  therefore  optimal. 

For  three  variations  on  the  basic  search  problem  described  in  the  first  section,  w?  nave 
developed  and  Implemented  algorithms  for  a Prime  400  minicomputer.  In  Chapters  n, 
ni,  and  IV  we  illustrate  the  type  and  complexity  of  problems  that  these  algorithms 
can  solve. 

In  the  first  section  of  this  chapter  we  describe  the  basic  class  of  moving  target 
problems  that  we  will  consider.  In  the  second  section  we  present  the  necessary 
and  sufficient  conditions  which  form  the  basis  of  the  algorithms  given  In  Chapters  n 
and  IV.  These  conditions  have  a very  interesting  and  useful  interpretation  in  terms 
of  optimal  search  for  stationary  targets  which  is  disoussed  In  the  second  section.  The 
third  section  describes  the  algorithm  used  to  compute  optimal  search  plans. 


Basic  Search  Problem 


Id  this  section  we  give  a description  of  the  basic  search  problem  which  is 
considered  throughout  most  of  this  report. 

Target  motion.  The  target's  motion  is  represented  by  a stochastic  process 
X = { X{;  t * 1, ....  T}  where  Xfc  is  a random  variable  which  gives  the  target's 
position*  at  time  t.  The  marginal  distribution  of  X(  is  simply  the  probability 
distribution  for  the  target's  location  at  time  t. 

In  a typical  search  problem,  one  is  given  the  target's  initial  probability 
distribution  and  a stochastic  description  of  the  target's  motion.  These  two  can 
be  combined  in  the  manner  discussed  in  Chapters  n and  IV  to  produce  the  stochastic 
process  which  represents  all  the  possible  target  paths  over  time  t = 1 T. 

As  an  example  one  might  have  a bivariate  normal  distribution  for  the  target's 
location  at  time  t = 1.  Such  a distribution  could  be  obtained  from  a long  range 
sensor  with  poor  localization  capabilities . From  geographical  considerations 
one  might  be  able  to  deduce  that  the  target's  course  lies  between  bearings  6 ^ and 
6 f and  from  operational  considerations  that  the  submarine  is  traveling  v/lth  a 
speed  between  v1  and  Vg.  Assume  that  the  probability  distributions  on  target's 
course  and  speed  are  uniform  over  the  above  limits . In  addition  one  could  assume 
that  the  target,  having  chosen  a beading  and  speed  from  the  above  distribution, 
persists  in  that  course  for  a random  time  which  is  exponentially  distributed  with 
specified  mean  r . At  the  end  of  this  time,  the  target  makes  a new  and  independent 
draw  from  the  above  distributions  to  determine  its  new  course  and  speed. 

* We  understand  to  be  either  a point  in  the  plan  or  a cell  index.  It  will  be 
clear  from  context  which  Is  the  case. 


Using  the  COMPASS  or  MEDSEARCH  programs,  this  type  of  target  motion  is 
represented  by  producing  a large  number  of  target  paths  (see  reference  (j  J)  in  a 
Monte  Carlo  fashion.  From  these  paths,  the  programs  can  generate  a sequence 
of  distributions  for  the  location  of  the  target  at  a sequence  of  times 
chosen  by  the  user.  The  target  distribution  at  time  t is  then  the  distribution  of  Xfc, 
and  the  collection  of  target  paths  represents  the  stochastic  process  X = {X  t * 1, . . . , 1}  . 

It 

Alternatively  this  type  of  target  motion  can  be  modeled  by  using  a Markov  chain  approach 
as  described  in  Chapter  Q.  The  target  motion  process  X then  becomes  a Markov 
chain.  The  target  motion  models  considered  in  this  report  and  the  algorithms 
discussed  below  are  capable  of  handling  both  of  these  possibilities  and  Indeed  a 
much  wider  range  of  target  motions.  In  fact  the  algorithm  in  Chapter  IV  will  find 
optimal  allocations  for  any  target  motion  process  which  can  be  represented  by  a 
Monte  Carlo  simulation. 

Search  grid.  For  most  of  the  report,  we  shall  assume  that  a grid  has  been 
established  on  the  search  space  which  is  two  dimensional  and  chat  we  must  allocate 
our  effort  uniformly  within  a given  cell  of  the  grid,  although  effort  may  vary  from 
cell  to  cell.  Let  J denote  the  set  of  cells  in  the  grid,  and  let 

Pt(J)  = Pr{Xtincellj} 

= Pr{  target  is  in  cell  J at  time  t}  for  Je  J,  t = 1, . . ,,T, 


and 


AO)  = area  of  1th  cell, 


for  jcJ. 
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FIGURE  1-1 
SEARCH  GRID 


cell  j 

Pt(J)  ■ Pr{  target  Is  In  cell  j at  time  t} 
A(J)  * area  of  cell 

J = collection  of  cells  In  the  grid 

Note:  The  search  grid  need  not  be  uniform. 
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Although  we  Impose  this  grid  structure,  the  target  motion  may  take  place  in  either 
discrete  or  continuous  space. 

Class  of  search  plans  considered.  A search  plan  is  described  by  a nonnegative 
function  ip  of  time  and  space  such  that 

p( j,t)  = effort  placed  in  cell  j at  time  t for  J e T,  t = 1, . . . ,T. 

We  assume  that  the  amount  of  effort  available  is  constrained  so  that 

m(t)  = total  effort  available  at  time  t. 

We  restrict  ourselves  to  search  plans  ip  such  that 

2 =m(t)  for  t *=  1 T.  fl-1) 

j€J 

The  class  of  plans  which  satisfy  (1-1)  is  called  ♦(m). 

Detection  function.  Let  u Indicate  a sample  target  path  drawn  from  the 
stochastic  process  {Xt;  t = 1, . . . , T}  . We  shall  let  X^uj)  be  the  target's  position 
(l.e.,  cell)  at  time  t on  this  sample  path.  Suppose  that  ip  is  a search  plan.  Then 
the  probability  of  detecting  the  target  given  it  follows  path  u is  assumed  to  be 


. - exp|- W(XtM)  ip  (Xt(w),  t)/A(XtM)J 


d-2) 


where  for  J e J,  W(j)  is  the  sweep  width  or  effectiveness  parameter  for  search  in  cell  J. 
That  is,  we  are  assuming  that  the  detection  function  is  exponential  and  that  detection 
of  the  target  during  one  time  period  is  Independent  of  detection  during  any  disjoint  time 
period.  In  Chapter  V,  first  section,  we  treat  the  case  where  the  probability  of 
detection  on  the  path  u>  is  given  by  b(z  ),  where  z is  the  effort  density  wbioh  accumulates 

UJ  U) 
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on  the  target  along  the  path  u>  and  b Is  a regular  detection  function,  i.e. , b has 


a positive  continuous  and  strictly  decreasing  derivative.  When  b is  not  exponential. 


detection  is  not  independent  from  time  period  to  time  period. 


Let  £ denote  expectation  over  the  sample  paths  of  { X^,  t = 1, . . . ,T}  . Suppressing 


the  dependence  of  Xt  on  w,  we  can  write  the  overall  probability  of  detection  by  time  T 


,1  * ] £ E jl-exp|-  W (X^  ip  (Xt,  Q/ApC^jj 


for  any  search  plan  ip . 


Problem  statement.  We  seek  a plan  ip*  which  satisfies  the  effort  constraint 


(1-1)  and  maximizes  the  probability  of  detection  by  time  T within  the  class  of  plans 


satisfying  (1-1).  Mathematically  stated,  we  seek  ip*  e 4»(m)  such  that 


PTl^*l  = PT(^J  : ip  € 'J'(m)}  . 


Such  a plan  is  called  T -optimal  within  *(m) . 


Basic  Necessary  and  Sufficient  Conditions 


In  this  section  we  present  the  basic  necessary  and  sufficient  conditions  which 


we  shall  use  in  constructing  the  algorithms  discussed  in  this  report.  The  form 


of  the  conditions  given  here  applies  to  discrete-time  target  motion  processes 


and  exponential  detection  functions.  A generalization  of  these  conditions  to  Include 


continuous  time  motion  processes  and  non-exponential  detection  functions  Is 


stated  and  proved  in  Chapter  VI. 


B 


In  order  to  state  these  conditions  we  let  if>  * denote  a plan  which  is  T-optlmal 


within  ¥(m)  and  let 


Itarget  Is  in  cell  j failure  to  detect  at  all  times 
) rjat  timet  J other  than  t using  plan  4>* 


for  jeJ,  t«l, . ..,T.  (1-4) 


Observe  that  the  conditioning  on  failure  in  (1-4)  Includes  search  which  takes  place 
after  time  t as  well  as  before.  Let  denote  expectation  conditioned  on  the  target 
being  in  cell  ] at  time  t.  Then  there  is  a constant  K(t)  for  t = 1, . . . , T such  that 

VC,t)  *^>E1tf*p(''8^W(X^*(Xft)/A(X^j|  for)€j'  t=1---T- 


The  constant  K(t)  is  simply  the  normalizing  constant  required  so  that 


jf"J  0»f)  = 1 for  t=l»  . . . »T. 


BASIC  CONDITION.  In  order  that  ip* e *(m)  be  T-optlmal  it.  is  necessary 
and  sufficient  that  the  allocation  <p  *(• , t)  maximize  the  probability  of 
detecting  a stationary  target  with  distribution  , t)  and  effort  m(t) 
for  t=l, . . .,T. 


To  show  that  if  <P  * Is  T-optlmal  within  Sk(m) , then  it  must  satisfy  the  basic 
condition,  we  reason  as  follows:  We  may  write  the  probability  of  failing  to  detect 
the  target  by  time  T using  <p*  as 
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l-PTi#*l  - E Texp J - 1 ^ W(X ^ f • I X t , t) /A (X^jj 

' jfj  Pt0)  Ejth>(-^  W<XS)  P*<XS.  »/A(Xb))]  e-W0>»*0.*)/Afl) 

= K(„  ?jg^,0.«)e-W(1)**0’t)/AU). 
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One  can  see  from  (1-5)  that  the  failure  probability  for  the  search  over 

time  [0,  T]  is  proportional  to  the  failure  probability  resulting  from  applying  the 
allocation  4>*(’  ,t)  specified  by  the  optimal  plan  for  time  t to  the  stationary  target 
problem  with  target  distribution  *(■ , t) . Thus  if  ip  *(• , t)  did  not  minimize  failure 
probability  (i.e. , maximize  detection  probability)  for  a stationary  target  with  this 
distribution  under  the  effort  constraint  m(t),  one  could  find  an  allocation  f such 
that  I^jfQ)  = m(t)  and 


Then  one  could  define 


*P*{ j,  S)  if  8 t t 
f(j)  if  s = t. 


Clearly  the  search  plan  w is  a member  of  *(m)  and  P^l^)  > PT[^*]  which  contradicts 
the  optimality  of  ^ *.  Thus  ip*(-  ,t)  must  maximize  the  probability  of  detection  for 
a stationary  target  with  distribution  ,t)  and  effort  m(t).  This  proves  the 
necessity  of  the  basic  condition.  A proof  of  the  sufficiency  is  given  in  Chapter  VI. 
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Description  of  Basic  Algorithm 


I 


t 

! 


i 

i 

i 


Most  of  the  algorithms  described  In  this  report  make  use  of  the  b&Bic  condition 
to  find  a sequence  of  search  plans  which  converge  to  the  optimal  plan.  They  usually 
proceed  as  follows. 

First  Iteration . For  time  t = 1,  find  the  optimal  allocation  of  search  effort  m(t) 

for  the  initial  target  location  distribution  p . Set  tp  V . 1)  equal  to  this  allocation 

and  compute  g*  the  posterior  target  distribution  at  time  2 given  failure  to  detect 

the  target  at  time  1.  Set  ip  1(* , 2)  equal  to  the  optimal  allocation  of  m(l)  effort  for 

the  distribution  g*  Suppose  that  we  have  found  tp  V , s)  for  s = l, . . . ,t-l.  Then  we 
£ 

cor  ‘inue  by  computing  g*,  the  posterior  target  location  distribution  at  time  t given 

xailure  to  detect  the  target  at  time  a = 1 t-1,  using  the  allocations  <p*(’ , s)  for 

s = 1 t-1.  Set  ip  V , t)  equal  to  the  optimal  allocation  of  m(t)  effort  to  the 

distribution  g*.  Continue  until  time  T is  reached.  This  constitutes  the  first  pass. 

The  allocation  tp 1 obtained  on  this  pass  Is  called  the  incrementally  optimal  or 

myopic  plan . That  is,  at  each  time  period  tp 1 allocates  its  effort  ir.  such  a way  as 

to  maximize  the  increase  in  detection  probability  for  that  time  period.  For  most 

moving  target  problems  the  myopic  plan  is  not  T-optimal  for  T > 1. 

2 

Second  Iteration.  Begin  th;s  iteration  by  computing  g , the  posterior  probability 

distribution  for  the  target's  location  at  time  t = 1,  given  failure  to  detect  the  target 

1 2 
at  times  t = 2, . . . , T,  using  $ {• , t)  for  the  allocation  at  time  t = 2, . . . T.  Set  ip  (• , 1) 

2 2 

equal  to  the  optimal  allocation  of  m(l)  effort  for  target  distribution  g^.  Usually  tp  (• , 1)  ^ 
1 2 

P (•,  1),  because  gj  differs  from  the  initial  target  distribution,  p , at  time  1.  If 

2 2 
ip  (• , s)  has  been  computed  for  s = 1, . . . , t-1,  then  find  ip  (•  ,t)  as  follows:  compute 


4 


j 

i 


a 
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2 

g the  posterior  target  location  distribution  at  time  t given  failure  to  detect  using 

2 1 
the  allocations  ip  (• , s)  for  s = i, . . . , t-1,  and  the  allocations  tp  (• , s)  for  8 « t+1, . . . , T, 

2 2 
and  set  ip  (.  (t)  equal  to  the  optimal  allocation  of  m(t)  for  the  distribution  g^.  Continue 

until  time  T is  reached.  This  constitutes  the  second  iteration. 

th  D 2 

n iteration.  Continue  as  in  the  second  iteration  with  >P  in  place  of  >P  and 

, n-1  . , . , 1 

>P  m place  of  >P  . 

The  above  algorithm  converges  to  the  optimal  plan.  That  is  one  can  find  a plan 
whose  probability  of  detection  is  as  close  to  optimal  as  he  desires  by  performing  a 
large  enough  number  of  iterations.  Notice  that  each  iteration  produces  a plan 
with  a higher  probability  of  detection  than  the  previous  one. 

The  above  algorithm  amounts  to  solving  a sequence  of  stationary  target  problems 
to  solve  an  optimal  moving  target  problem.  There  are  very  efficient  algorithms  for 
solving  stationary  target  problems  when  the  detection  function  is  exponential.  (See 
Example  2.2.8  of  reference  [h |.)  Thus  we  can  produce  efficient  algorithms  which 
follow  the  procedure  outlined  above  provided  we  can  compute  the  posterior  target 
distributions  g^,  t = 1, . . . , T,  n = 1,  2, . . . , efficiently.  Computation  of  these 
distributions  will  depend  on  the  target  motion  model,  i.e.,  the  nature  of  the  stochastic 
process  { X^,  t = 1, . , . ,T}  . For  example,  the  algorithm  described  in  Chapter  n 
is  designed  for  Markovian  target  motions . In  this  case  there  are  very  efficient 
methods  for  computing  the  distribution  g°.  The  algorithm  described  in  Chapter  IV 
can  be  used  with  any  Monte  Carlo  target  motion.  This  algorithm  is  more  flexible 
than  the  one  for  Markovian  motions  but  it  is  also  slower  and  requires  an  additional 
approximation,  i.e. , that  involved  in  replacing  the  stochastic  process  { Xt>  t = 1, . . . ,T} 
by  some  large  but  finite  number  of  its  sample  paths. 
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CHAPTER  II 


MU LTISCENARIO  MARKOVIAN  MOTION 


In  this  chapter*  we  consider  the  special  case  of  the  basic  search  problem  in 
which  the  target  moves  within  a finite  number  of  cells  in  discrete  time.  The  target's 
motion  process  is  assumed  to  be  multiscenario  Markovian  which  means  that  the  target 
is  assumed  to  be  following  one  out  of  N possible  motion  scenarios  and  each  scenario 
is  modeled  by  a discrete  time  and  space,  finite  Markov  chain. 

In  the  first  section  we  discuss  the  class  of  BearcheB  that  will  be  considered  in 
this  chapter.  In  the  second  section  we  present  examples  of  optimal  plans.  The 
third  section  describes  the  algorithm  used  to  compute  the  optimal  plans  and  proves 
that  it  converges  to  the  optimal  plan.  The  fourth  section  discusses  a method  of 
producing  constrained  Markovian  motions. 


Target  Motion  Model 

The  target  is  assumed  to  be  following  one  of  N motion  scenarios.  The  probability 

th 

that  the  target  is  following  the  n scenario  is  a where 

n 


£ a = 1. 

i n 
n=l 


♦This  chapter  is  based  on  reference  (b  ]. 


Each  aoenarlo  is  modeled  by  a Markov  ohala  in  the  following  manner.  Let  (1  j© 
the  set  of  possible  target  paths  over  times  t = 1,  . . . , T.  A member  « of  ft 
specifies  the  sequence  of  oells  that  the  target  follows  from  time  1 to  T,  and 

U)  ” • • • » U^)i 

! 

th  n 

where  a;  Is  he  cell  that  contains  the  target  at  time  L For  the  n scenario,  q (u>), 
the  probability  that  target  follows  path  us,  is  given  by 

qn(w)  = r^ojj)  Tn(o>2,o>3)  ...  r“ u>T). 

where  r “(i,  j)  is  the  probability  that  a target  located  in  cell  1 at  time  t will  transition 
to  cell  j at  time  t + 1 and  rn(j)  is  the  probability  that  the  target  starts  in  cell  J at 
time  1 under  scenario  n.  Thus  each  scenario  corresponds  to  a Markov  chain  with 
transition  probabilities  which  may  vary  with  time. 

The  distribution  rD  specifies  the  target  distribution  at  time  1 for  scenario  n. 

It  is  also  possible  to  specify  the  target's  distribution  at  any  additional  time  t = 1,  . . . , T. 

This  has  the  effect  of  modifying  the  transition  probabilities  t n for  the  scenario.  The 
method  of  accomplishing  this  is  discussed  in  the  fourth  section.  It  results  in  path 

probabilities  of  the  form 

1 

n , n/  * n,  , n . . n . n . . /TT  - 

q (6>)  = p 1(o>1)t  jlojj.ojglPgCwg)  . ..  T T-l^T-l’^T  P T^l**  U_1 

Thus,  we  are  considering  the  search  problem  described  in  the  first  section  of 
Chapter  I restricted  to  the  case  where  the  target  motion  process  { X^;  t*l, . . . , T}  is  a 
discrete  time  and  space,  multiscenario,  constrained  Markovian  prooess. 
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Examples 


We  now  give  three  examples  of  optimal  plans  found  by  the  algorithm  described 

in  the  third  section.  In  all  cases  J is  a subset  of  a grid  of  square  cells  with  sides 

three  miles  long  which  are  oriented  north-south  and  east-west.  One  cell  is  chosen 

and  labeled  (0,  0).  The  cell  which  is  i cells  west  and  j cells  north  of  (0,  0)  is  labeled  (i,  j). 

Each  search  lasts  for  eight  time  intervals  of  one  hour  each.  We  take  W(i,  j)  = 1 for  all 

cells  (i,  j)  so  that  z units  of  search  effort  placed  in  the  cell  which  contains  the  target 

—2  /9 

will  yield  a detection  with  probability  1-e 

The  transition  function  t is  chosen  to  model  truncated  triangular  distributions 
on  course  and  speed.  When  r is  independent  of  time,  the  subscript  t will  be  suppressed. 
We  specify  the  speed  distribution  by  giving  the  minimum  speed,  maximum  speed,  best 
speed,  and  a weight  (3.  The  target  chooses  its  speed  from  the  truncated  triangular 
distribution  in  Figure  II— 1.  The  density  of  the  best  speed  is  (3  times  the  density  of 
the  minimum  speed,  the  latter  equals  the  density  of  the  maximum  speed.  When  = 1 this 
is  a uniform  distribution.  A similar  distribution  describes  the  target's  course.  Let 
T ((i>  j)>  (k>  l ))  be  the  probability  that  a target  which  starts  at  the  center  of  cell  (i,  j) 
and  which  chooses  its  course  and  speed  from  the  given  triangular  distribution  moves 
to  cell  (k,  l ) in  one  hour. 

Example  1:  Markovian  fan.  The  target's  initial  distribution  is  circular  normal 
with  a standard  deviation  of  6 miles.  It  is  centered  in  the  middle  of  cell  (0,  0)  and 
truncated  by  a 15  mile  by  15  mile  square  whose  center  coincides  with  the  center  of  the 
distribution.  The  orientation  of  the  square  matches  the  grid.  The  target's  motion  is 
single  scenario  Markovian  with  the  target  choosing  its  course  uniformly  from  150° 
to  210°  and  its  speed  between  6 and  12  kts  with  a best  speed  of  9 kts,  which  is  given  a 
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FIGURE  II - 1 


TRIANGULAR  DISTRIBUTION  ON  TARGET  SPEED 


weight  of  p * 1. 5.  Independent  draws  are  made  fro\n  these  distributions  each  hour. 

This  type  of  moving  target  problem  Is  similar  to  that  faoed  by  an  aircraft  trying  to 
redetect  a submarine  on  which  contact  has  been  recently  lost. 

Table  II-l  Bhows  how  the  target  density  changes  with  time  In  the  absence  of 
search.  The  target  moves  southward  and  diffuses  from  its  original  distribution.  It 
stays  within  a sixty  degree  wedge  and  has  a oentral  tendency  because  there  are  more 
paths  which  lead  to  the  center  of  the  wedge  than  to  the  edges. 

We  assume  W(j)  * 1 for  je  J.  Ninety  units  of  search  effort  are  available  each 
hour.  Table  11-2  gives  the  final  detection  probabilities  achieved  by  the  first  ten  Iterations 
of  the  optimization  algorithm  described  below.  The  detection  probabilities  converge 
very  rapidly:  Indeed  the  first  approximation  to  the  plan,  the  myopic  plan.  Is  quite 
satisfactory.  Table  II-3  compares  the  detection  probabilities  achieved  by  the  myopic 
plan  at  the  end  of  each  hour  to  those  achieved  by  the  plans  which  are  optimal  for  2 
through  8 hours.  During  the  beginning  of  the  search,  the  myopic  plan  outperforms  the 
plans  which  are  optimal  for  later  times.  In  no  case  is  the  myopic  plan  significantly 
outperformed  by  any  optimal  plan. 

Table  11-4  shows  the  myopic  plan  and  the  optimal  plan  for  eight  hours  oriented 
with  north  at  the  top  of  the  page.  The  myopic  plan  starts  by  putting  most  of  its  effort 
In  the  center  of  the  target  distribution.  It  diffuses  and  moves  southward  with  the  target. 
The  optimal  plan,  on  the  other  hand,  starts  by  using  most  of  its  effort  to  surround 
the  target.  By  the  end  of  the  search  the  optimal  plan  closes  in  on  the  target.  At 
hour  eight  the  optimal  plan  puts  70  percent  more  effort  In  the  center  of  the  target's 
distribution  than  the  myopic  plan  does. 
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TABLE  II -1 

TARGET  DENSITY  FOR  THE  MARKOVIAN  FAN 

Notes:  (1)  Entries  represent  thousandths  of  a target. 

i2)  Ceils  are  3 mi  x 3 ml. 
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Example  2:  Constrained  Markovian  fan.  In  this  example  we  consider  a target 


motion  which  is  identical  to  the  one  in  Example  1 except  that  the  target  is  constrained 
to  have  a truncated  normal  distribution  at  time  8 which  is  equal  to  the  initial  distribution 
translated  60  miles  due  south. 

The  transition  function  t for  this  constrained  process  is  computed  by  the  method 
discussed  in  the  fourth  section. 

Table  II— 6 shows  how  the  target  density  changes  over  the  first  four  hours  in 
the  absence  of  search.  The  target  moves  southward  and  diffuses  from  its  original 
distribution.  During  the  last  four  hours  the  target  density  is  a mirror  image  of  the 
first  four  hours.  It  still  mwes  south  but  converges  back  to  a truncated  normal 
distribution  with  a 6 mile  standard  deviation. 

Ninety  units  of  search  effort  are  available  each  hour.  The  detection  probabilities 
for  successive  Iterations  of  the  algorithm  are  . 926,  . 932,  . 932,  . 932,  and  . 932 
which  repeats.  These  probabilities  converge  rapidly  and  again  the  first  approximation 
to  the  plan,  the  myopic  plan,  is  quite  satisfactory.  Table  II -6  compares  the  detection 
probabilities  achieved  by  the  myopic  plan  at  the  end  of  each  hour  to  those  achieved 
by  the  plans  which  are  optimal  for  2,  4,  6,  and  8 hours.  Again,  no  optimal  plan 
significantly  outperforms  the  myopic  plan.  With  the  exception  of  Example  3 below, 
these  conclusions  have  been  confirmed  by  other  examples  including  some  with  much 
smaller  overall  detection  probabilities. 

Table  II— 7 shows  the  optimal  plan  for  eight  hours.  Again,  only  the  tables  for 
the  first  four  hours  are  displayed  sinoe  the  last  four  hours  are  a mirror  image  of 
them.  At  hour  1,  the  optimal  plan  uses  most  of  its  effort  to  surround  the  target  rather 
than  searching  the  highest  probability  cells  as  the  myopic  plan,  which  is  shown  at  the 
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TABLE  1 1 - 5 

TARGET  DEN8ITY  FOR  THE  CONSTRAINED  MARKOVIAN  FAN 


Notes:  (1)  Entries  represent  thousandths  of  a target. 
(2)  Cells  are  3 mi  by  3 mi. 

Hour  1 


2_ 

_1_ 

_0_ 

^2 

2) 

23 

34 

38 

34 

23 

1] 

34 

49 

56 

49 

34 

0] 

38 

56 

63 

56 

36 

-1) 

34 

49 

56 

49 

34 

-2] 

23 

34 

38 

34 

23 

Hour  2 
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3 

2 

1 

0 

^1 

^2 

^3 

^4 

0J 

0 

1 

3 

7 

7 

7 

3 

1 

0 

-1] 

0 

6 

17 

31 

36 

31 

17 

6 

0 

-21 

1 

9 

27 

50 

58 

50 

27 

9 

1 

-3] 

1 

10 

30 

55 

64 

55 

30 

10 

1 

-4] 

1 

9 

25 

47 

54 

47 

25 

9 

1 

~5] 

0 

4 

12 

23 

26 

23 

12 

4 

0 

-6] 

0 

0 

2 

3 

3 

3 

2 

0 

0 

Hour  3 


4 

3 

2 

i 

0 

2l 

zl 

^3 

-2] 
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0 

1 

1 

1 

1 

1 

0 

0 

“31 

0 

2 

5 

9 

11 

9 

5 

2 

0 

-4] 

2 

7 

18 

31 

37 

31 

18 

7 

2 

-5J 

2 

11 

29 

49 

59 

49 

29 

11 

2 

-6] 

3 

12 

31 

53 

63 

53 

31 

12 

3 

“7] 

2 

9 

24 

40 

48 

40 

24 

9 

2 

-8J 

1 

4 

10 

17 

21 

17 

10 

4 
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“9] 

0 

1 

2 

3 

3 

3 

2 

1 

0 
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TAB  LE  1 1 - S (continued) 


Hour  4 
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8 

2 

0 

-8] 
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30 

50 

60 

50 

30 

12 

3 

0 

r- 

-9] 

0 

3 

12 

31 

52 

61 

52 

31 

12 

3 

0 

V * 
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0 

2 

9 

21 

36 

43 

36 

21 

9 

2 

0 

-11] 

0 

1 

3 

8 

14 

17 

14 

8 

3 

1 

0 

-12] 

0 

0 

1 

1 

2 

3 

2 

1 

1 

0 

0 

Hours  5-8  are  a mirror  reflection  of  hours  1-4. 


TABLE  11-6 

DETECTION  PROBABILITIES  FOR 
THE  CONSTRAINED  MARKOVIAN  FAN 


N umber  M yopic  Plan  which  is  optimal  for 

of  hours  plan  2 hours  4 hours  6 hours  8 hours 


i 

.357 

.346 

.324 

.313 

.311 

2 

. 555 

.560 

.549 

.541 

.640 

3 

.672 

.679 

.673 

.672 

4 

. 750 

.762 

.758 

.767 

•- 

5 

. 807 

.818 

.817 

6 

. 852 

.862 

.862 

7 

. 889 

.898 

• * 

8 

.926 

.931 

1 
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TABLE  1 1 - 7 


SEARCH  PLANS  FOR  THE  CONSTRAINED  MARKOVIAN  FAN 

2 

Notes:  (l)  Entries  represent  thousandths  of  a unit  of  search  density  (effort/ml  ) 
(2)  Cells  are  3 ml  by  3 ml. 

Optimal  Plan  for  8 Hours 


Hour  1 
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^1 

^2 
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448 

563 

568 

563 

448 

11 

423 

361 

263 
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0 
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0 

^1 

^2 

23 

-4 

-2J 

0 

0 

0 

0 

0 

0 

0 

0 

0 

-31 

0 

0 

0 

0 

0 

0 

0 

0 

0 

-4] 

0 

0 

257 

449 

484 

449 

257 

0 

0 

-5] 
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0 
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0 
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TABLE 

II-7  (continued) 
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Hours  5 

-8  are  a mirror  reflection  of  hours  1-4. 

5 

Myopic  Plan 

1 

Hour  1 

i: 

i 

u 

2 

1_ 

0 

^1 

^2 

2] 

0 

256 

381 

256 

0 

i 
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Hours  2-3  not  shown. 
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0 
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401 
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0 

92 
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Hours  5-8  not  shown. 
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end  of  Table  11-7,  does.  By  hour  4,  however,  the  optimal  plan  concentrates  its 
effort  in  the  center  of  the  target  distribution. 

Example  3:  Multiple  scenario  motion.  Next  we  consider  a multlscenarlo  example. 
Figure  11-2  Illustrates  the  two  possible  scenarios  for  target  motion  with  (1,  j)  Indicating 
the  midpoint  of  cell  (l,  j).  In  scenario  1 the  target  starts  uniformly  distributed  over 
the  square  in  Figure  11-2  which  is  northwest  of  the  obstruction.  It  proceeds  southward 
to  distribute  Its  density  at  hour  4 uniformly  over  the  square  southwest  of  the  obstruction. 
Within  the  limits  Imposed  by  these  constraints,  the  target  chooses  courses  and  speeds 
from  the  triangular  distribution  given  in  row  I of  the  table  at  the  bottom  of  Figure  11-2. 
After  hour  4 the  range  of  courses  and  speeds  widens  as  given  in  row  II.  In  scenario  2 
the  target  again  starts  in  the  northwest  rectangle  but  now  It  goes  around  the  obstruction 
to  the  north  and  east.  Uniform  distributions  over  the  squares  shown  in  Figure  2 are 
Imposed  on  the  target  location  distribution  at  hours  1,  4,  and  7.  The  course  and  speed 
distributions  that  hold  between  hours  1-3,  4-6,  and  7-8  are  given  by  rows  III,  IV,  and 
V. 


Search  effort  becomes  available  gradually  during  the  eight  hours  of  the  search. 

No  search  Is  available  for  the  first  two  hours.  Nine  units  of  search  effort  are 

1 

available  during  each  of  hours  3 and  4 and  36  units  of  search  effort  are  available 
during  each  of  hours  5 through  8.  The  final  detection  probabilities  obtained  by 
successive  iterations  of  the  algorithm  are  . 466,  .519,  .619,  .519,  and  . 519  which 
repeats.  After  a jump  between  the  first  and  second  iteration,  the  convergence  Is 
characteristically  rapid. 

Table  II— 8 compares  the  detection  probabtlltes  for  the  myopic  plan  with  the 
optimal  plan 8 for  4 through  8 hours.  For  8 hours  the  optimal  plan  improves  the 
detection  probabilities  from  .465  to  . 519.  The  qualitative  difference  in  the  plans  , 1 
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FIGU_R_£_IJ-2_ 

GEOGRAPHY  FOR  MULTISCENARIO  MOTION 

Note:  Staging  areas  are  nine  miles  square, 
w Cells  are  three  miles  square. 

A 


Hour  1 

/ 


Hour  4 

\ 45+ 

V 


15  miles' 


24  miles 
Obstruction 


Hour  4 


Hour  7 


Scenario  1 

Scenario  2 

Course 

Speed  (knots) 

x.  best  weight 

min.  max.  best  weight 

TABLE  II - 8 


DETECTION  PROBABILITIES  FOR  THE 
MULTISCENARIO  MOTION 


Number 
of  hours 

Myopic 

plan 

4 hours 

Plan  which  is  optimal  for 

5 hours  6 hours  7 hours 

8 hours 

3 

.063 

. 062 

.058 

.041 

.042 

.042 

4 

.112 

. 112 

.097 

.078 

.078 

.078 

5 

.256 

.269 

.262 

.262 

.252 

6 

.361 

.382 

.380 

.376 

7 

.425 

.519 

8 .465 
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which  causes  this  Increase  Is  shown  in  Table  II-9.  Because  the  second  scenario  has 
higher  weight,  the  myopic  plan  searohes  it  almost  exclusively  during  the  early  hours. 
During  the  later  hours,  the  first  scenario  diffuses  so  widely  that  the  second  scenario 
is  still  searched  almost  exclusively.  The  myopic  plan  is  handicapped  because  it  does 
not  look  ahead  to  see  that  first  scenario  targets  get  away  quickly  and  search  this  scenario 
while  it  is  still  possible.  The  optimal  plan  does  this.  It  searches  the  first  scenario 
until  it  starts  to  diffuse,  then  shifts  to  the  second  scenario.  This  strategy  is  optimal 
for  the  long  terra  but,  as  Table  11-8  shows,  involves  a penalty  during  hourB  3 and  4. 

Further  examples  of  optimal  search  plans  computed  by  these  techniques  can 
be  found  in  reference  (a]. 

Description  of  Algorithm 

Recall  from  the  discussion  in  the  third  section  of  Chapter  I that  the  basic  step 
in  the  optimization  algorithm  Is  a reallocation  of  search  effort  at  a single  time  interval. 
Example  2. 2.  8 of  reference  [hj  gives  an  algorithm  for  finding  an  optimal  allocation  of 
search  effort  for  a stationary  target  in  a discrete  search  space  when  the  detection 
function  is  exponential.  Using  the  necessary  conditions  in  Corollary  2. 1. 6 of 
reference  [h]  one  can  show  that  the  optimal  allocation  is  unique. 

Suppose  that  we  have  a search  plan  ip.  Let  g^(*  ,t)  be  the  posterior  target 
location  distribution  at  time  t given  failure  to  detect  at  all  times  other  than  t using 
the  plan  ty.  Let  f*  be  the  optimal  allocation  of  m(t)  effort  for  the  stationary  target 
problem  with  distribution  g^(- , t).  Define  Ht*/'  to  be  the  search  plan  that  is  obtained 
from  4>  by  replacing  the  allocation  at  time  t with  the  allocation  f*.  That  is,  for  Jc  J, 


S iMJ.s) 


for  s ¥ t 
f*(j)  for  s = t. 
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SEARCH  PLANS  FOR  MULTISCENARIO  MOTION 


1 


Notes:  (1)  Entries  represent  thousandths  of  a unit  of  search  density  (effort/mi2). 

(2)  O indicates  the  position  of  the  obstruction. 

(3)  Cells  are  3 mi  x 3 mi. 


Myopic  Plan 


Optimal  plan  for  8 hours 


Hour  1 — No  Search  Available 
Hour  2 — No  Search  Available 
Hour  3 


0 0 0 0 286  251 

0 0 0 0 246  217 

0 0 0 0 0 0 

0 0 0 0 0 0 

0 0 0 0 0 0 

0 0 0 0 0 0 


0 0 0 
0 0 0 
0 0 0 
0 0 0 
204  281  0 

211  304  0 


0 

0 

0 

0 

0 

0 


i 


Hour  4 


ooooo 
0 0 0 0 0 
ooooo 
ooooo 
ooooo 
000  0_0 
0 0 0 0 0 
9 9 9 0 0 
9 9 9 0 0 


0 46 

0 202 

0 223 

0 0 

0 0 

0 0 

0 0 

0 0 

0 0 


0 

231 

245 

0 
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0 

0 

0 

0 


0 

0 

0 

0 

0 

0 

0 

154 

157 


0 0 

0 0 

0 0 

0 0 

0 0 

0 0 

0 0 

92  266 

94  237 


0 0 0 0 0 

0 0 0 0 0 

0 0 0 0 0 

0 0 0 0 0 

ooooo 
0o0  000 
0^0  000 
ooooo 
ooooo 


s 


Myopic  Plan 


Tj^BLE  11-9  (conMnuedL 

i Optimal  plan  for  8 hours 


Hour  5 


0 0 0 0 0 241  0 

0 0 0 0 464  718  675 

0 0 0 0 321  714  730 

0 0 0o0  0 140  0 

0 0 0^0  000 

0 0 0 0 0 0 0 

0 0 0 0 0 0 0 

0 0 0 0 0 0 0 

0 0 0 0 0 0 0 

0 0 0 0 0 0 o 


0 0 324  112 

0 287  645  569 

0 165  654  563 

0 0 217  0 

*0  0 0 0 


24  150  111  151  24 

0 0 4 0 0 


Hour  6 


0 0 

0 

201 

94 

0 0 

0 

279 

130 

0 

526 

548 

627 

0 0 

395 

631 

581 

0°0 

471 

556 

682 

0°0 

345 

657 

596 

0 0 

0 

161 

133 

0 0 

0 

279 

107 

1 

Hour  7 

i 

oOo 

261 

82 

214 

o 

0 

o 

353 

0 

248 

0 0 

537 

316 

556 

0 0 

611 

79 

538 

0 0 

525 

312 

594 

0 0 

614 

110 

603 

0 0 

224 

114 

264 

0 0 

356 

90 

398 

Hour  8 


0°0  0 0 0 

0 0 0 0 0 

0 0 203  428  204 


0°0  0 0 0 

0 0 0 0 0 

0 0 152  427  153 


0 0 376  609  376  0 0 420  711  421 

0 0 376  609  377  0 0 392  674  392 

0 0 59  326  58  0 0 0 259  0 


The  algorithm  proceeds  as  follows: 

(1)  Let  be  an  Initial  guess  for  the  search  plan. 

(2)  Let  e be  a small  number. 

(3)  Let  l = 0. 

(4)  Perform  step  (5)  for  t = 1 to  T. 

(5)  Set  V'T/+t  * st^TM-l4 

(6)  If  | PT1^(/+1)X1  - PTI^iTll  < «.  stop:  the  answer  Is  * M)T. 

(7)  Otherwise  Increase  l by  1,  and  return  to  step  (4). 

While  the  initial  guess  may  be  any  search  plan,  we  generally  use  j,t)  = 0 
for  j € J,  t * 1,  ....  T.  In  this  case  fonr  ‘.ng  ip^,  ip^,  ....  ip  corresponds  to 
allocating  Bearch  effort  at  successive  time  intervals  In  order  to  obtain  the  greatest 
Increase  In  at  the  current  time  interval  — not  to  maximize  P^  In  the  long  term. 
Thus,  is  the  myopic  or  Incrementally  optimal  plan. 

Implementation  of  the  algorithm.  The  only  difficult  step  in  the  algorithm  Is  the 
computation  of  H.-  Let  ip  = Then  this  step  consists  of  computing  the 

t t I +t-l 

distribution  g^(- , t)  and  finding  an  optimal  allocation  of  m(t)  effort  for  this  distribution 
when  the  detection  function  Is  exponential.  Example  2. 2.  8 of  reference  [h]  gives  an 
algorithm  for  finding  such  an  allocation,  so  we  consider  only  the  computation  of 

Since  the  algorithm  for  finding  an  optimal  allocation  for  g^(- , t)  produces  the 
same  allocation  for  any  "distribution"  Kg^(- , t),  where  K is  a positive  constant,  we 
shall  be  concerned  only  with  calculating  g^  In  an  unnormalized  form. 
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For  je  J,  define 

R0.1  .#>  *=  PjO). 

and  for  l<t<T,  let 

R(j.t,i/0  = S R(k,t-1,  ip)  exp(-W(k)tf(k,t-l)/A(k))T  (k,J)p  (J)  • 
ke  J 1 

Similarly  for  je  J,  let 

S(j,T,  4>)  = 1 

and  for  1 < t < T, 

S(j.t ,4>)~  S t (j,k)p  (k)  exp(-W(k)^(k,t+l)/AfltP0t.t+l,  ip). 
keJ 

If  Pj  Is  the  initial  distribution  and  pt(j)  = I for  t>l  and  je  J,  then  R and  S have 
natural  probability  Interpretations.  In  fact,  R(j,t,  ip)  is  the  probability  that  the 
target  reaches  cell  j at  time  t and  is  not  detected  by  the  effort  at  times  Ml,  ....  t-1 
while  S(j.  t,  4>)  Is  the  probability  that  if  the  target  starts  In  cell  j at  time  t It  will 
not  be  detected  by  the  effort  at  times  s = t+1,  . . . , T.  As  a result,  R(J,  t,  ip) S(j,t,  ip) 


Is  the  probability  that  the  target  is  located  In  cell  J at  time  t and  is  not  detected  by 


of  R and  S remains  true  for  the  weights  p constructed  for  the  constrained  Markovian 
motion.  Thus,  g^(* , t)  may  be  computed  by  the  above  formula  for  constrained  Markov 
motion.  As  we  noted  before,  the  algorithm  in  Example  2.2. 8 of  reference  [h]  is 
Insensitive  to  multiplication  of  the  target  distribution  by  a constant,  so  the  factor 
K(t)  is  not  computed. 

Observe  that  it  is  not  necessary  to  recompute  the  entire  R and  8 functions  at 

each  step  in  optimization.  Suppose  one  has  reaohed  time  T in  the  algorithm  and  Is 

about  to  begin  at  time  1 again,  i.  e. , one  has  returned  to  step  (4)  in  the  algorithm 

description.  The  allocation  just  computed  is  ip  ^ for  some  i.  At  this  point  one 

computes  S(- , t,  ip  ) recursively  for  t = 1,  . . . , T-l,  calculates  g (. , 1)  from  (II-4) 

’'ZT 

and  reallocates  effort  to  obtain  ^^T+1-  Having  done  this  one  computes  R(*  ,2, 

from  01-2)  and  then  g from  (U-4)  using  the  fact  that  S(- , 2,  ip  ) - 8(- , 2,  ?,_) 

^ZT+1  11  1 

because  the  reallocation  of  effort  at  time  1 does  not  affect  the  computation  of 

8(* . 2,  iPlT+.  )•  The  distribution  g is  computed  in  a similar  fashion  for  t * 2,  . . . , T. 

Thus  for  one  cycle  through  the  time  periods  t ■ 1,  . . . , T,  one  need  compute  R and  8 

only  once. 

If  there  is  more  than  one  scenario,  then  one  proceeds  as  above  to  compute 

n th  N 

g (• , t)  for  the  n scenario  and  takes  g , (• , t)  * £ a g°  (■ , t). 

ip  "if, 

n*x 

The  computer  program  which  Implements  this  algorithm  is  described  in 
references  [1  ] and  fcn). 

Convergence  of  the  algorithm.  We  now  show  that  this  algorithm  halts,  that 

# # 
the  resulting  plan  ip  is  in  'i'(m)  and  that  as  c approaches  zero,  ip  approaches 

& 

optimality.  The  definition  of  Hfc  shows  that  ip  e ♦(m)  and  that  0 < 

- PXl*1)  - • ‘ ^ ThuB  llm^co  PTl^  eXl8tS  “d  llmZ-*»  I PTl1z+l)TJ  " PT^ZT^  ^ * °* 


It  follows  that  the  algorithm  halts.  As  t -»0,  the  answer  is  farther  into  the  sequenoe 


f , ....  so  we  need  only  show  that  lim^  { ) = max{ PT(*)  : i>  e *(m)} . 
In  fact,  we  shall  show  that  for  some  subsequence  4>*  m llmk^.ao  ^ exists 

and  is  T-optimal  within  4'(m). 

Since  J and  T are  finite  and  m(t),  t *=  1,  ....  T is  bounded,  we  may  consider 
if,  1 = 1,  2,  ...  to  be  ft  sequence  of  points  in  a bounded  set  of  a finite  dimensional 

CD 

Euclidean  space.  Thus,  there  le  a subsequence  {*k>kB:1  »uch  that  : k»l,  2,  . 

k 

converges  to  a point  tp*.  Clearly  ip*  e *(m).  The  continuity  of  PT  implies  that 

llm/  * Um^P^^  ) * PTl^*)  so  that  we  need  only  show  that 

k 

Plli*]  = max{  PT(01  : ip  £ *(m)}  , i.e. , ip*  is  T-optimal  within  *(m). 

Since  lim.  exists  and  PT  and  Ht  are  continuous  for  t - I,  ....  T,  it 

follows  that 


t 


| 

1 

j 

l 

i 

i 


vi  - itok  -wj = 


- llm^  PX[E  T))  - PTtSj(^)).  1 

k j 

* 

I 

8ince  PTIVJ*1  * Pt(E1^’*]i  it  follows  that  both  ip*{ • , 1)  and  » 1)  are  optimal  j 

allocations  for  the  stationary  target  problem  with  distribution  g(- , 1)  and  exponential 

1 

J 

detection  function.  By  the  uniqueness  of  such  solutions  E.  4>*  * <P*>  By  a similar  ‘ 

* | 
argument  E„^.  \p*  *=  E„  $ + and,  in  fact,  E.  ip*  = ip*  for  t * 1,  ....  T.  i 

Thus,  for  each  t - 1,  . . . , T,  4>*(- , t)  maximizes  the  probability  of  detection  for  a 

stationary  target  with  distribution  g (•  ,t)  and  exponential  detection  function  under  -i 

■3 

the  effort  constraint  m<t).  8o  ip*  satisfies  the  basic  necessary  and  sufficient  condition 


given  in  the  second  section  of  Chapter  I and  is  therefore  T-optimal. 
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\ 

i 
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Constrained  Markovian  Motion 


In  this  section  we  discuss  a method  which  allows  the  user  to  specify  constraining 

distributions  for  the  target's  location  at  times  t , t , ....  t € { 1 T} . Normally, 

the  search  planner  specifies  an  initial  distribution  and  a transition  matrix  to  identify 

a Markov  process  whioh  represents  the  target's  motion.  The  initial  distribution  may 

be  obtained  from  a long  range  detection  with  poor  localization  and  the  transition 

matrix  may  be  developed  from  general  knowledge  of  the  target's  behavior.  However, 

the  search  planner  may  have  information  which  leads  him  to  believe  that  the  target 

will  have  known  distributions  r for  1 = 1,  . . . , n where  1 < t < . . . < t < T.  In  this 

t(  - 1 n ~ 

section  we  discuss  a method  for  modifying  a given  transition  function  t to  obtain  a 

Markov  chain  {X  ; t = 0,  ....  T}  such  that  X , the  target's  position  at  time  t , has 

i 1 

distribution  r for  1 = 1,  . . . , n. 

To  begin,  we  consider  the  situation  in  which  the  target's  distribution  is  specified 
at  time  t = 1 and  t = T.  (For  this  discussion  we  will  assume  that  there  is  only  one 
motion  scenario  and  drop  the  scenario  index  from  r,  t , and  q. ) We  consider  target 
path  probabilities  q(o>)  which  are  computed  by 

q(w)  = P(wl)T1(aj1,aj2)T2(a)2,aj3)  . ..  7T_1<wT_1'«T)P'(wT>»  (U“5) 

where 

p(j),  p'O)  >0  for  J € J. 

In  most  cases,  p and  p'  will  not  be  probability  distributions.  If  p is  a probability 
distribution,  then  equation  (1-5)  may  be  Interpreted  as  taking  a Markovian  motion 


with  prior  distribution  p and  transition  matrix  t and  combining  this  according  to 
Dempster's  rule  (see  references  [n}  and  [o])  with  an  independent  estimate  of  the 
target  position  at  time  T which  is  represented  by  the  probability  distribution  which  is 
proportional  to  p\  The  information  In  p'  will,  in  general,  change  our  estimate  of 
where  the  target  was  at  time  zero.  It  will  also  ohange  our  estimate  of  how  the  target 
made  its  transitions.  If,  for  example,  p'  is  a more  concentrated  distribution  than 
the  distribution  of  a target  which  diffuses  from  p according  to  t , then  the  target's 
transitions  for  times  near  T will  have  to  have  a bias  toward  the  mean  of  p'  to  recon- 
centrate the  target's  density. 

In  order  to  compute  the  prior  distribution  and  transition  probabilities  which 
result  from  this  constrained  model,  we  Introduce  the  functions  G and  H where 
G:  {1 T}  x J x 1]  is  defined  recursively  by 

G(l.J,j)  « l,G(l,i,J)  = 0 ifi^j 

G(t,  i,  j)  - E G(t-l,i,k)r  (k,J)  for  2 < t < T 
ke  J 

and  H:{l T}  xJx  J— (0, 1J  is  defined  recursively  by 

H(T,J,J)  - 1,  H(T,i,J)  - 0 ifi/j 

H(t,  i,  j)  = 2 t (l,k)  H(t+l,k.J)  for  1 < t < T-l. 
ke  J 

For  a Markov  chain  governed  by  the  transition  function  t , G(t,  i,  j)  is  the  probability 
that  the  chain  is  in  state  j at  time  t given  it  was  in  state  i at  time  1.  Similarly, H(t,  i,  J) 
is  the  probability  that  the  chain  is  in  state  j at  time  T given  it  was  in  state  i at  time  t. 
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Let 


r(l)  * S p(l)H(l.l.J)P,0)  *<»r  IcJ. 

From  tie  deftolttoa  of  B sad  tie  pati  probsblUtlee  1.  one  osa  eee  that  r(l)  la  the 
gym  of  tie  probabtllttee  of  all  pathe  which  atari  at  I,  so  r(l>  Is  tie  probailUty  that 
the  target  starts  to  cell  1.  Likewise,  the  probailUty  that  tie  target  eade  to  oell  1 

is  given  by 

r'(j)  = Z P 0)  H(l.t.J)  P'O). 

U J 

The  imposition  of  the  weights  p and  p*  on  the  sample  path  probabilities  causes 
the  transition  probabilities  of  the  Markov  chain  to  be  modified.  Let  r indicate  the 
modified  transition  function  and  {X<  ; t = 0,  . . . . T>  the  stochastic  process  with  path 
probabilities  given  by  q in  (II-5).  Then 

?t0,l>  * Pr{Xt=lsadXt+1=)}/Pr{Xt.t}, 

where 

2 

Pr{  Xt  = l and  Xt+J  * j}  * {w.a>t=i  and  * )}  d(o>) 


(II-lO, 


(ii-n, 


(U-12)j 


= 2 2 p(k)  G(t,k,l)  T t(i.  j)  H(t+l,J,k’)  P*(k'), 

k€  J k'c  J 


and 


Thus 

Vi,j)  = T (l,J)  r 2 H(t+l,J,k*>  p'(k')“|  / r 2 H(t,i,k')  p'(k'f|  . 

1 1 Lk’eJ  J j J 

From  equations  (U-10)  and  (0*13)  one  can  calculate  that 

T-l  . 

Pr{w}  « r(w  ) n t fa  , o>  , ) 

i 1 * 1 x 

T-l  . 

= p(wj)  n Ttfat.o>t+1)  p'faT) 
t~l 

= q(u>). 

Thus  the  transition  function  t along  with  the  Initial  distribution  r produces  the  path 
probabilities  q defined  In  <11— 5). 

The  search  planner's  estimate  of  the  target's  local  motion  characteristics  is 
normally  In  terms  of  r . When  the  searcher's  estimate  of  the  target's  prior  distri- 
bution comes  from  a contact  with  an  associated  uncertainty,  which  is  based  on  the 
characteristics  of  the  detector  and  the  detectability  of  the  target,  but  not  on  the 
subsequent  motion  of  the  target,  then  this  is  an  estimate  of  p . On  the  other  hand, 
when  the  searcher's  estimate  of  the  target's  prior  distribution  comes  from  historical 
information  which  accounts  for  the  target's  subsequent  motion,  then  this  is  an 


estimate  of  r.  Thus  the  searcher  may  reasonably  start  with  either  r or  p . Likewise, 
he  may  start  with  either  r'  or  p'. 

Tne  program  described  in  reference  [a]  handles  four  forms  of  input;  p , t , p'; 

P , t , r';  r,  t , p ';  and  r,  t , r'.  Computationally,  the  form  p , t , p ' is  easiest  to 
work  with  so  we  convert  the  other  Inputs  to  this  form.  When  p,  t , and  r'  are  entered, 
we  use  equations  (II-8)  and  (II-9)  to  compute  H and  then  solve  equation  (11-11)  for  p*. 


1 

J 


I 

j 

) 

A 


The  solution  is  just 


P'O)  * r'(j)/  2 p(k)H(l,k,J) 
ke  J 

for  all  je  J.  The  denominator  of  this  expression  can  vanish  only  when  it  is  physically 
impossible  for  a target  which  starts  in  a cell  k with  p 0c)  > 0 and  makes  only  transitions 
which  r gives  nonzero  probability  to  get  to  cell  j at  time  T.  If  r'(j)  also  vanishes,  we 
can  set  p'(j)  = 0.  On  the  other  hand,  if  r'(j)  4 0,  then  the  inputs  describe  an  impossible 
situation  and  no  solution  Is  possible.  Likewise  when  r.  t , and  p'  are  entered,  we 
compute  H and  then  solve  equation  (II— 10)  for  p . Again  there  is  no  solution  only  in 
a physically  impossible  situation. 

The  final  input  possibility;  r,  t,  and  r' ; presents  greater  difficulties.  We 
compute  H as  above  and  then  solve  the  quadratic  equations  (11-10)  and  (11-11)  for 
p and  p'.  This  is  accomplished  by  the  following  iterative  algorithm: 

(1)  Let  c be  a small  positive  number. 

(2)  For  all  j e J set  pQ(j)  = r(j). 

(3)  For  all  j t J set  p' (j)  = r'(J). 

V f 

(4)  Set  / * 0. 

(5)  For  all  J € J set  p ^(j)  = r(j)/  H(l,  J,  k)  pj  (k)J  . 

(6)  For  all  J e J set  pj  (!)  = r'(j)  / [ Z p <k>  H(l,k,  j)j  . 

kc  J 

(7)  If  I P/+1(J)  - p j(J)  I < e and  |p ]+1(k)  - pj(k)|  < € for  all  j,  kc  J.  then 
stop:  The  answer  is  p = p^+1  and  p ' = p|+J. 

(8)  Otherwise,  set  1 = 1- *1. 

(9)  Go  back  to  step  (5). 


As  above,  a aero  should  be  supplied  for  the  result  of  an  Indicated  division 
of  zero  by  zero  in  steps  (5)-(6).  While  no  convergence  results  are  available  for 
this  algorithm,  it  has  proven  effective  In  examples  suoh  as  those  given  in  the 
second  section. 

More  generally,  we  may  have  Information  about  the  target's  distribution  at 

a sequence  of  time  Intervals  1 < t < t < . . . < t < T.  For  this  case  we  consider  a 

— 12  n — 

more  general  definition  of  q as  follows: 

T-l 

q(c*j)  = p1(^1)  n [T t^t* a5t+i)  pt+i^t+i^ 
t — l 

For  t i t,,  ....  t„  we  take  p4(j)  = 1 for  je  J.  For  t , n = 1 I,  p^  is  a non- 

l t t n t 

n 

negative  function  on  J which  is  related  to  our  information  about  the  target's  distri- 
bution at  the  times  tQ,  n = 1 1.  That  is  the  user  may  specify  p^  for  tQ  = 1,  ....  I 

or  he  may  specify  rfc  the  target's  distribution  at  time  t and  solve  for  p as  above 
n n n 

for  n = 1,  ....  1. 

Once  we  have  solved  for  or  obtained  the  P for  t - 1,  . . . , T,  we  can  calculate 
q(a>)  as  though  it  were  obtained  from  a Markov  chain  with  initial  distribution  Pj  and 
transition  function  x'  defined  by 

Tt(i,J)  = Tt(i,J)  pt+l(J)  for  t = 1 T-l  and  i,  Je  J, 

even  though  p^  may  not  be  a probability  distribution  and  t'  may  be  a defective 
transition  function.  This  causes  no  problem  because  the  probabilities  computed  in 
this  fashion  are  all  proportional  to  the  correct  probabilities,  ,’n  particular  the  target 
location  distribution  at  any  t is  proportional  to  the  one  obtained  from  this  Markov 
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ohaln.  The  algorithm  will  give  the  same  allocations  of  effort  for  any  two  distributions 
whioh  differ  by  a constant  factor. 

Our  final  observation  concerning  the  motion  model  is  that  it  can  be  used  to  create 
multiple  scenarios.  Each  state  can,  for  example,  oonsist  of  a oell  and  a scenario 
index.  The  target  whose  state  oonsists  of  oell  j and  scenario  n moves  by  applying 
a transition  matrix  associated  with  scenario  n to  J to  obtain  ltB  new  cell.  It  abvays 
keeps  the  same  scenario.  The  Initial  scenario  weights  are  incorporated  in  the  amount 
of  target  mass  which  the  prior  distribution  on  the  states  places  in  states  with  each 
scenario  index. 
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CHAPTER  III 


RECTANGULAR  SEARCH  PLANS 


In  this  chapter  we  discuss  rectangular  search  plana,  which  model  sonobuoy 

fields,  to  search  for  a target  moving  in  discrete  space  and  in  discrete  time  with 

a fixed  time  limit,  T.  The  computer  algorithm,  which  generates  the  rectangular 

search  plans,  is  described  and  the  effectiveness  of  these  rectangular  plans  is 

measured  by  comparison  (via  computed  examples)  with  T-optimal  plans.  In  the 

last  section  of  this  chapter  we  summarize  outstanding  problems  associated  with 

rectangular  barrier  search.  All  of  the  examples  in  this  chapter  are  taken  from 
references  [c]  and  [d). 

The  Search  Problem 

The  search  space,  search  time,  and  target  motion  assumptions  are  as  in  Chapter 
II  with  the  exception  that  at  each  time  Interval  the  searcher  must  spread  his  limited 
amount  of  effort  uniformly  over  a rectangular  region.  The  probability  of  detecting 
the  target  during  the  t**1  time  Interval  if  rectangle  R is  searched  is  given  by 

[1-  exp(-m(t)/A(R))]  • p(R) 

where 

m(t)  = total  effort  available  at  time  t 
A(R)=  area  of  the  rectangle  R 
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p(R)  = probability  that  the  target  Is  located  in  R. 


The  number  of  time  intervals,  T,  is  specified  at  the  outset  and  the  searcher  attempts  to 
choose  a rectangle  to  search  at  each  time  interval  to  maximize  the  probability  that 
he  will  detect  the  target  within  the  specified  time  limit.  We  refer  to  an  allocation 
of  effort  which  accomplishes  this  objective  as  an  optimal  rectangular  plan  to 
distinguish  it  from  the  T-optlmal  plans  found  in  Chapter  II.  When  the  number  of 
time  intervals  is  one,  we  refer  to  the  problem  of  choosing  an  optimal  rectangle  as 
a rectangular  stationary  target  problem. 

Rectangular  Barrier  Algorithm 

In  this  section  we  describe  the  computer  algorithm  which  computes  the  rectangular 
search  plans  given  in  the  examples  in  the  next  section.  Although  we  do  not  claim  that 
our  rectangular  plans  are  optimal  rectangular  plans,  experimental  comparison  with 
T-optlmal  plans  shows  that  our  plans  cannot  be  far  from  rectangular  optimality  since 
in  fact  these  plans  are  not  far  from  T-optimality. 

The  motivation  for  the  approximations  made  by  our  algorithms  may  be  found  in 
reference  [p],  where  the  problem  of  allocating  a fixed  amount  of  effort  uniformly 
over  a rectangle  to  find  a stationary  target,  whose  location  distribution  is  bivariate 
normal,  is  considered.  Suppose  the  bivariate  normal  distribution  has  starciard 
deviations  and  a 2 along  the  major  and  minor  axes,  respectively.  Reference  ( p] 
then  considers  only  rectangles  centered  at  the  mean  of  the  distribution,  oriented 
along  the  major  axis  of  the  distribution,  and  whose  length  and  width  are  proportional 
to  Oj  and  cr2,  respectively.  The  resulting  one  variable  problem  can  be  solved  by 


standard  calculus.  Comparison  of  the  detection  probabilities  of  the  search  plan 
thus  produced  to  the  totally  optimal  detection  probability  shows  that  the  approxima- 
tions involved  are  extremely  good  (see  reference  [q]).  ; 

‘•£ 

With  the  above  motivation  in  mind,  we  can  now  state  our  rectangular  barrier  f 
algorithm.  Our  algorithm  follows  the  description  of  the  basic  algorithm  as  given  in 
the  third  section  of  Chapter  I,  except  that  it  attempts  to  solve  a rectangular  stationary 
target  problem  at  each  time  step  instead  of  an  ordinary  stationary  target  problem. 

This  being  the  case,  we  need  only  describe  the  workings  of  our  algorithm  at  a fixed 
time  step  on  the  rectangular  stationary  target  problem  which  results  from  motion  as 
well  as  failure  to  detect  at  all  other  time  periods. 

The  rectangular  stationary  target  problem  is  a problem  in  maximizing  a function 
of  five  variables  given  by  the  length,  width,  orientation,  and  center  of  the  rectangle. 

In  general,  this  function  need  not  be  convex,  so  a straightforward  attempt  to  find 
the  maximum  by  computer  algorithms  would  be  difficult.  Instead,  we  try  to  take 
advantage  of  the  intuition  exhibited  in  reference  [p]  to  make  some  good  choices  for 
some  of  the  variables,  thereby  reducing  the  number  of  variables  in  the  optimization. 

Thus,  at  each  time  in  the  search  we  proceed  as  follows.  Check  whether  the  distri-  4- 

| 

button  is  unimodal  or  not.  If  it  is  unimodal  we  consider  a single  class  of  rectangles,  | 

% 

while  if  the  distribution  is  multimodal  we  consider  three  classes  of  rectangles.  f 

-S 

in  the  unimodal  case  we  restrict  ourselves  to  the  class  of  rectangles  that  are  centered  I 
at  the  mean  of  the  distribution  and  oriented  along  the  major  axis  of  the  distribution.  | 
The  major  axis  of  the  distribution  is  obtained  by  computing  the  covariance  matrix  of  | 
the  distribution  and  finding  the  angle  of  rotation  that  diagonalizes  the  matrix.  This 
correspond.'  tv  the  rotation  of  coordinates  which  transforms  a normal  distribution 

1 
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with  that  covariance  matrix  Into  the  product  of  two  Independent  one-dimensional 
normal  distributions  along  the  rotated  axes.  The  axis  having  the  distribution  with 
the  largest  variance  Is  the  major  axis.  We  then  optimize  over  the  two  remaining 
variables — length  and  width. 

In  the  multimodal  case,  we  consider  the  above  class  of  rectangles  and  two 
additional  classes.  To  determine  the  additional  classes  we  compute  the  two  best 
modes  of  the  distribution.  We  then  mimic  . was  done  In  the  uni  modal  case  for 
the  distribution  near  each  of  these  modes.  To  do  this  we  assign  cells  to  be  In  the 
"local  distribution"  of  one  of  these  modes  as  follows.  Proceed  radially  from  the 
mode  until  the  probabilities  begin  to  Increase.  Cut  of!  the  "local  distribution"  at 
this  point.  The  resulting  grid  of  ceLls  Is  what  we  call  the  "local  distribution"  of 
that  mode.  We  now  renormalize  these  "local  distributions"  and  compute  the  moan 
and  covariance  matrix  of  these  two  "local  distributions.  " This  leads  to  two  further 
classes  of  rectangles,  and  proceeding  In  the  manner  described  above,  we  find  the 
best  rectangle  In  each  of  these  classes.  We  now  have  three  "best"  rectangles,  one 
based  on  ea^h  of  the  two  best  modes  and  one  centered  at  the  mean  of  the  distribution. 
Finally,  we  take  the  best  rectangle  from  among  these  three  best  rectangles. 

Examples 

in  this  section  we  discuss  three  examples  of  rectangular  search  plans. 

Example  It  Markovian  fan,  Tn  our  first  example,  the  target  motion  Is  identical 
to  that  In  Example  1 ot  Chapter  II. 

Table  1 below  show:,  the  target  density,  updated  by  scarob  effort,  and  our 
rectangles  for  8 hours,  Tht  target  moves  southward  and  diffuses  from  Its 
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PROBABILITY  MAP  AND  RECTANGLE  FOR  HOUR  5 


JMM  COP*  *>  ^ 


TAB  LE  III  - 1 (continued) 


PROBABILITY  MAP  AND  RECTANGLE  FOR  HOUR  6 
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original  distribution.  The  searcher  has  90  units  of  search  effort  available  each  hour 
which  he  applies  uniformly  over  the  rectangles  shown. 

As  can  be  seen  from  Table  m-1,  the  rectangles  chosen  for  all  8 time  periods 
are  Intuitively  very  reasonable.  Table  III-2  below  gives  the  detection  probabilities 
for  each  time  interval  for  our  rectangular  plan  and  optimal  plan  for  eight  hours. 

As  Table  m-2  shows,  the  rectangular  plan  has  detection  probability  only  . 017 
less  than  the  detection  probability  of  the  totally  optimal  plan. 

Example  II:  constrained  Markovian  fan.  We  now  modify  the  previous  example 
by  assuming  that  historical  Information  indicates  the  target's  distribution  at  time 
8 as  well  as  at  time  1.  In  particular,  we  assume  that  at  hour  8 the  target's  prior 
distribution  is  identical  to  the  initial  distribution  but  translated  60  miles  south.  The 
motion  in  this  example  is  identical  to  that  in  Example  2 of  Chapter  il  Note  that  the 
distribution  which  appears  In  Table  III-3  below  Is  not  circular  normal  at  time  8 
because  search  effort  has  been  applied  during  hours  1 through  7 Inside  the  1*  Heated 
rectangles. 

For  each  of  hours  1 through  8, Table  m-3  shows  the  target  distribution  conditioned 
on  failure  to  detect  prior  to  that  time  along  with  the  search  rectangle  for  that  hour. 

The  target  initially  diffuses,  but  by  hour  6 It  must  start  to  reconcentrato  its  density 
to  attempt  to  meet  the  constraint  at  hour  8, 

Because  of  the  constraint  at  hour  8,  the  posterloi  distributions  are  more 
concentrated  so  that  guessing  the  reotanglee  by  inspection  might  appear  to  be  easy. 
However,  choosing  by  Inspection  can  lead  to  mistakes.  For  instance,  one  might 
guess  that  at  hour  2,  the  5 x 8 rectangle  with  vertioes  (2,-1),  (2,-6),  (-2,-6),  and 
(-2,-1)  Is  a better  rectangle  tha-  die  one  choeen  by  the  algorithm.  However,  one 
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i.  TA_BLE_iUr2_ 

DETECTION  PROBABILITIES  FOR  EXAMPLE  1 

Probability  of  Detection  for 

• • 

Optimal  Plan 

I Time  Rectangular  Plan  for  8 Hours 
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TABLE  IH-3 

PROBABILITY  MAPS  AND  RECTANGULAR  SEARCH  PLAN 
FOR  EXAMPLE  2:  CONSTRAINED  MARKOVIAN  FAN 


Notes:  (1)  Entries  represent  thousandths  of  a target 

(2)  The  reotangle  for  hour  1 is  the  square  which  surrounds  the 
entire  distribution. 

(3)  Cells  are  3 ml  by  3 ml 
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PROBABILITY  MAP  AND  RECTANGLE  FOR  HOUR  2 
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TABLE  I1I-3  (contlp u_e d_^ 
PROBABILITY  MAP  AND  RECTANGLE  FOR  HOUR  5 
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PROBABILITY  MAP  AND  RECTANGLE  FOR  HOUR  8 
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can  check  that  the  probability  of  detection  us  In*  this  rectangle  is  . 52194,  which  Is 
not  as  good  as  the  rectangle  chosen  by  the  computer. 

Table  in-4  compares  the  detection  probabilities  for  the  optimal  and  rectangular 
plans  for  each  of  the  eight  hours.  As  can  be  seen,  the  rectangular  plan  never  is 
significantly  behind  either  the  myopic  or  optimal  plans.  The  rectangular  plan  has  a 
final  detection  probability  only  . 010  less  than  the  optimal  plan  for  eight  hours. 

Example  3:  Radial  flee.  In  our  last  example  the  target  starts  from  the  same 
prior  distribution  used  in  the  two  previous  examples.  Its  speed  is  chosen  uniformly 
from  the  interval  (6, 12].  To  choose  its  course,  it  first  chooses  one  of  eight  base 
headings:  22.50,  67.5°,  112.5°,  157.5°,  202.5°,  247.5°,  292,5°,  or  337.  5°. 
Having  chosen  one  of  these  base  headings  it  keeps  it  for  the  entire  five  time  units 
of  the  search.  Once  a base  heading  is  chosen  the  target's  course  at  each  time 
interval  is  chosen  uniformly  from  whichever  of  the  following  Intervals  contains 
the  base  heading:  [0°l45°],  [45°,  90°],  [90°,135°],  [135°,  180°),  [180°,  225°], 

[225°,  270°],  [270°,  315°],  or  [315°,  360°].  This  example  approximates  the 
classical  radial  flee  target  motion  discussed  in  reference  (i). 

As  can  be  seen  In  Table  III-5  the  target  diffuses  very  rapidly  and  for  hours  3,  4, 
and  5 the  target's  distribution  is  clearly  multimodal.  The  rectangles  chosen  during 
these  times  are  clearly  those  from  the  class  of  rectangles  corresponding  to  one 
of  the  two  best  modes.  Table  III-6  compares  the  rectangular  plan  and  the  optimal 
plan  for  each  of  the  5 time  intervals. 

Outstanding  Problems 

In  this  section  we  summarize  outstanding  problems  associated  with  the 


rectangular  barrier  problem. 


TABLE  III  - 5 


RECTANGULAR  PLAN  FOR  RADIAL  FLEE 


PROBABILITY  MAP  AND  RECTANGLE  FOR  HOUR 
Note:  Entries  represent  thousandths  of  a target. 
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PROBABILITY  MAP  AND  RECTANGLE  FOR  HOUR  2 
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TABLE  III-J5  (continued) 


PROBABILITY  MAP  AND  RECTANGLE  FOR  HOUR  5 
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There  are  two  main  problems  wh4ch  need  further  work.  The  first  is  to 


Incorporate  a more  realistic  detection  function,  which  Is  better  suited  to  the 
sonobuoy  problem.  The  second  Is  to  produce  algorithms  which  will  allow  the 
user  to  pick  rectangles,  which  are  to  stay  in  place  for  more  than  one  time  period. 

As  a step  toward  solving  the  second  problem,  we  consider  the  problem  of 
finding  optimal  plans  which  are  restricted  to  have  a fixed  allocation  for  t hours 
before  choosing  another  allocation  for  the  next  t hours. 

As  in  Chapter  II  we  let  12  be  the  set  of  possible  paths  of  a target  moving  in 
discrete  space.  We  assume  that  the  searcher  has  tT  time  periods  to  search  and 
let  J be  the  set  of  all  possible  cells  through  which  the  paths  pass.  Because  of  the 
restriction  mentioned  above,  a search  plan  is  a function  ip  : J x { 1, . . . ,T  } — [0,  ®) 
such  that 

(j . t)  = effort  placed  In  cell  j at  times  t (t-l)  + 1, ...  ,Tt. 

If  a search  plan  4 also  satisfies 

2 <[  (j,  t)  = mft)  for  t = 1, . . . , T, 
je  J 

then  we  say  4 t ♦(m).  Note  that  we  have  m(t)  amount  of  effort  available  for  each 
of  the  times  t (t-l)+l, . . . , t t.  The  only  restriction  is  that  we  must  allocate  the 
effort  in  the  same  way  for  each  of  these  time  periods. 

Given  a search  plan  4 we  assume  that  the  detection  function  is  exponential 
so  that  the  probability  o.  detection  using  4 is  given  by 

T T-l 

^ q(a-)  exp -2  2 W(WTt_.)  iffw,  t_«,  t)/A(o>  ) , 

? ^S!  [t=l  i=0  Tt_l 

where  q(a')  is  the  probability  of  the  path  x-. 
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The  problem  iy  to  maximize  PT  j(i^J  subject  to  £11-1). 

Now  let  JT=  =0i jT)  | jjt' J for  1 = 1 t and  J1i  j2  . . .<  jT}  . For 

any  f e £T,  we  let 

nt<f ) - : | u>efi  ! u)  passes  through  the  ceils  in  J in  anjr  order  j 
during  times  t (t-l)+l, .... t.  ) 

For  pny  search  plan  ip,  f e and  t define 

T-l 

v(f,t,  ip)  = 2 . q(u))  exp{- 2 2 W(u  ) (u>  ,,  8)/A(u>  - . 

) S=/t  i=0  T8-i  TS-1  T8  1 

Then  for  each  t,  1<  t<  T,  we  may  write 

a T_1 

p TW-  1-  vO  . t,^)  exrf-,^  WO,)  ♦Oj.^/Aflj)}. 

j e J 

For  a search  plan  tp , we  let  be  given  by 

^t( J)  = ^O.t)  for  JcJ. 

For  a search  plan  ^ and  1 <t<T  we  define 

A T "I 

Q,HM  = 1-^r  expf-^  W0,,t)  fo,)} 

where  f:J-*-[0,«>i. 

We  can  now  characterize  an  optimal  search  plan  >p  *c  *(m) . 

THEOREM:  p*  e <Hm) maximizes  P „ over  <Pe  'J'(m)  if  and  only  if  for  each  t, 

" T T “ 

l<t<T, 

*t*]  = max{  Qt[^*,f):  2JcJfO)  = m(t)}  . (III-2) 
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Proof.  Since  both  P ^ and  * J are  concave  functions  and  the  effort 

constraints  are  linear,  the  Kuhn-Tucker  conditions  are  necessary  and  sufficient 
for  ip*  to  maximize  PtT  over  ip*  4»(m)  as  well  for  tp*  to  satisfy  (III-2).  In  fact, 
the  Kuhn-Tucker  conditions  are  the  same  for  both  problems,  and  thus  we  obtain 
the  equivalence  stated  above.  This  proves  the  theorem. 

Because  of  the  above  theorem,  it  is  clear  that  an  algorithm  for  computing  an 
optimal  plan  restricted  to  fixing  effort  every  r times  units  can  be  obtained  by 
following  a procedure  similar  to  that  given  in  Chapter  n when  t = l.  The  only 
difference  is  that  maximizing  • ] is  not  a standard  stationary  target  problem 

as  in  the  case  t = 1.  As  noted  above,  •)  is  a concave  function  and  the 

constraint  in  (III-2)  is  .1  linear  constraint'  thus,  we  can  apply  a nonlinear 
programming  approach  to  solving  this  problem. 
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COMPARISON  OF  RECTANGULAR  PLAN 
WITH  THE  OPTIMAL  PLAN  FOR  RADIAL  FLEE 


Probability  of  Detention  for 


Time 

Rectangular  Plan 

5-Interval 
Optimal  Plan 

1 

.329 

.357 

2 

.411 

.435 

3 

.437 

.465 

4 

.454 

.487 

5 

.468 

.505 
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CHAPTER  IV 


ARBITRARY  DISCRETE-TIME  TARGET  MOTION 

In  this  chapter  we  consider  optimal  search  for  a target  with  arbitrary  motion 
In  discrete  time.  We  discuss  and  give  examples  of  a method  for  finding  T -optimal 
search  plans  under  the  following  conditions: 

(1)  The  target  motion  is  modeled  by  a discrete  time  stochastic  process 
that  can  be  simulated  on  a computer.  This  process  may  take  place 
In  continuous  or  discrete  space. 

(2)  At  each  time  t,  m(t)  effort  is  to  be  applied  to  searching  for  the  target. 

(3)  The  detection  function  is  exponential  but  may  vary  over  space. 

(4)  A grid  is  specified  and  allocations  are  required  to  be  uniform 
within  the  cells  of  the  grid. 

Observe  that  the  class  of  allowable  target  motions  is  very  broad.  For  example,  the 
target  is  not  restricted  to  move  among  a set  of  cells  as  Is  usually  the  case  when 
one  deals  with  Markov  chain  models  of  target  motion  (as,  for  example,  in  Chapter  II). 

The  motion  process  need  not  be  Markovian  or  even  a mixture  of  Markovian  processes. 

The  process  can  be  Gaussian,  a constrained  dlffuslon,or  a random  movement  through 
a network.  In  fact,  the  optimizer  described  here  could  be  coupled  with  the  COMPASS, 
MEDSEARCH,  or  TARDI8T  programs  to  find  optimal  allocations  of  search  effort  over  any 
Interval  of  Interest  for  any  target  motion  processes  produced  by  these  programs. 

The  algorithm  proceeds  by  obtaining  a Monte  Carlo  sample  of  target  paths  and 
then  optimizing  over  that  sample.  Thus  the  accuracy  of  the  optimization  will  be 


related  to  the  accuracy  with  which  the  sample  represents  the  target  motion  process. 

In  addition  the  algorithm  considers  only  Bearch  allocations  which,  during  any  one 
time  period,  are  oonstant  within  the  cellB  of  a grid  chosen  by  the  user.  However, 
tb’.b  restriction  is  not  equivalent  to  assuming  that  the  target  motion  takes  place  in 
discrete  space.  The  algorithm  is  based  on  the  necessary  and  sufficient  conditions 
stated  in  Chapter  I. 

While  the  algorithm  described  applies  to  a more  general  class  of  motion 
processes  than  the  one  described  in  Chapter  II,  it  does,  in  effect,  optimize  over  a 
finite  sample  of  the  target  motion  process,  whereas  the  algorithm  in  Chapter  U 
treats  the  target  motion  exactly.  Thus  in  the  case  where  a mixture  of  a small  number 
of  discrete  time  and  space  Markov  processes  provides  a good  representation  of  the 
target  motion,  the  algorithm  in  Chapter  II  will  be  more  accurate  and  faster. 

In  the  first  section  we  give  examples  of  optimal  plans  computed  by  the  algorithm 
which  Is  described  in  the  second  section.  In  the  third  section  we  prove  the  algorithm 
converges  to  an  optimal  plan. 

Examples 

In  this  section  we  present  three  examples  of  optimal  plans  found  using  the 
algorithm  described  in  the  second  section. 

Example  1:  Radial  flee  from  s normal  distribution.  For  this  example  we  as- 
sume that  the  target  has  been  detected  by  a sensor  with  poor  localization  so  that 
its  initial  distribution  Is  circular  normal  with  standard  deviation  20  miles  in  any 
direction.  The  target  is  kncwn  to  be  moving  at  10  knots  on  a constant  but  unknown 
course.  We  assume  that  the  course  is  chosen  from  a uniform  distribution  on  0°  to  360°. 
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This  target  motion  is  an  example  of  the  classic  fleeing  target  motion  given  in 
reference  (l  J,  page  17.  That  reference  gives  the  probability  distribution  pt  for  the 
target’s  location  at  time  t given  do  search . In  Chapter  7 of  reference  [i] 
reasonable  search  plans  are  suggested  for  this  problem  but  in  the  years  since  that 
reference  was  prepared,  no  one  has  been  able  to  find  optimal  plans  for  this  problem. 

We  assume  that  the  searcher,  an  aircraft,  does  not  arrive  on  scene  until 
hour  4.  The  dete~'  on  function  is  given  by 

x . e-Wz/AQ)^ 

where  z is  the  number  of  sonobuoy  hours  placed  in  the  cell j and  W = 270  (ml)2/sono- 
buoy-hour.  This  detection  function  approximates  the  probability  of  detection  that 
one  obtains  from  search  by  sonobuoys  and  is  based  on  fitting  an  exponential  function 
to  a simulation  of  submarine  detections  by  sonobuoy  fields  as  discussed  in 
reference  (r).  The  constant  W = 270  (mi)2/3onobuoy-hour,  corresponds  to  good 
sonar  conditions.  We  assume  that  16  sonobuoys  are  available  and  that  the  search 
continues  for  4 hours.  Because  of  the  time  required  to  place  the  sonobuoys  In  the 
water,  we  assume  that  on  the  average  only  16  x.  75  sonobuoys  are  available  for  each 
hour  of  search.  Since  the  constant  W = 270  (ml)2/sonobuoy-hour  Is  the  same  over  all 
space,  we  shall  simply  absorb  this  into  the  effort  available  at  each  hour  and  assume 
that  there  are  3240  units  of  effort  available  each  hour. 

Table  IV- 1 shows  the  myopic  search  plan  for  this  problem  for  hours  4 through  7. 
The  target  distributions  shown  for  each  hour  are  the  distributions  at  that  time  given 
failure  of  the  previous  search  effort  to  detect  the  target.  Observe  that  at  time  4 
the  myopic  allocation  concentrates  its  effort  most  heavily  In  the  center  cells  of  the 


86 


TABLE  I V-  1 


RADIAL  FLEE  FROM  A NORMAL  DISTRIBUTION 


Myopic  Plan  — Hour  4 
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Myopic  Plan  — Hour  5 
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TABL  E _ 1Y  - 1 _(C_o_n_t_i  _n  u e d_l 
Myopic  Plan  — Hour  6 
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target  distribution,  but  at  times  S through  7 effort  is  concentrated  in  an  annulus 
centered  at  the  mean  of  the  target  distribution.  Except  for  variations  caused  by 
sampling  error  in  the  underlying  target  distribution,  the  effort  distribution  is  Been 
to  be  fairly  symmetric.  Table  IV-2  shows  the  optimal  plan  for  this  example.  For 
hours  4-7  the  cells  in  which  the  optimal  plan  places  a larger  amount  of  effort  than 
the  myopic  plan  are  indicated  by  shading.  Notice  that  the  optimal  plan  places  sub- 
stantially more  (e.  g. , 51  percent  more)  effort  in  the  center  four  cells  at  hour  4 than 
the  myopic  plan.  At  times  5-7  the  optimal  plan  appears  to  be  more  concentrated 
than  the  myopic  plan.  In  addition  one  can  see  that  at  time  5,  the  optimal  plan  chooses 
to  allocate  effort  heavily  to  the  eastern  side  of  the  distribution  and  then  balances  this 
by  heavy  allocations  on  the  western  side  at  times  6 and  7. 

Although  the  optimal  plan  is  qualitatively  quite  different  from  the  myopic  plan, 
one  can  see  from  Table  IV-3  that  there  is  very  little  difference  in  the  detection 
probabilities  from  these  two  plans. 

Example  2:  Radial  flee  from  a uniform  distribution.  In  this  example  the 
search  assumptions  are  exactly  as  in  Example  1 except  that  the  target's  initial 
distribution  is  uniform  over  a square  80  miles  on  a side.  Table  IV-4  shows  the 
myopic  plan  and  Table  IV-5  shows  the  optimal  plan  for  this  example. 

Since  the  initial  distribution  is  uniform  rather  than  normal,  one  might  have 
expected  that  both  the  optimal  and  myopic  plans  would  stop  searching  in  the  center 
of  the  distribution  earlier  than  in  Example  1.  However,  by  comparing  the  myopic 
and  optimal  allocations  at  time  5 for  this  example  to  those  in  the  previous  example, 
one  can  see  that  both  plans  and  in  particular  the  optimal  plan  continue  to  search 
the  center  of  the  distribution  longer  when  the  initial  'distribution  is  uniform  than 
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RADIAL  FLEE  FROM  A NORMAL  DISTRIBUTION 
Optimal  Plan  — Hour  4 

Note:  Shading  indicates  the  cells  in  which  the  optimal  plan  places 
more  effort  than  the  myopic  plan. 
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T _A_B_L  E_I_V  ^ 2 JJ3o_n  ti  n u_e 
Optimal  Plan  — Hour  5 

Note:  Shading  indicates  the  cellB  In  which  the  optimal  plan  places 
more  effort  than  the  myopic  plan. 
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Optimal  Plan  — Hour  6 

Note:  Shading  indicates  the  cells  in  which  the  optimal  plan  allocates 
’ more  effort  than  the  myopic  plan. 
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Optimal  Plan  — Hour  7 


Note:  Shading  indicates  the  cells  in  which  the  optimal  plan  allocates 
more  effort  than  the  myopic  plan. 
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TABLE  IV- 3 


PROBABILITY  OF  DETECTION  FOR  OPTIMAL  AND  MYOPIC  PLANS 
FOR  RADIAL  FLEE  FROM  A NORMAL  DISTRIBUTION 
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Myopic  Plan  — Hour  6 


TARGET  DISTRIBUTION 

NOTE*  ENTRIES  HAVE  BEEN  MULTIPLIED  BY  104 


0 

1 

7 

19 

41 

53 

35 

8 

0 

0 

>M. 

. 0 

3d 

66 

94 

126 

1 12 

108 

65 

22 

1 

1 1 

59 

1 00 

99 

95 

101 

103 

103 

72 

12 

3d 

95 

98 

95 

82 

88 

94 

1 1 1 

96 

29 

) 

i4 

1 1 7 

102 

92 

67 

73 

81 

86 

130 

3g 

3d 

TTCT 

"TT 

~JT 

73 

1 3 

TT 

0} 

ITT 

5^ 

40 

107 

101 

90 

80 

87 

93 

105 

86 

33 

}s . 

1 2 

62 

109 

IOd 

102 

95 

101 

109 

89 

Id 

r o 

27 

64 

93 

1 2d 

1 18 

85 

66 

24 

2 

0 

° 

I-UL 

33 

41 

44 

44 

/ 

1-1- 

0 

1°W 


1°E 


MYOPIC  ALLOCATION 


. o 

0 

45 

161 

113 

too 

0 

0 

S ' 

0 

70 

66 

49 

73 

83 

81 

0 

51 

61 

49 

0 

17 

45 

1 12 

54 

132 

-12- 

-21. 

0 

0 

0 

18 

175 

107 

61 

0 

6 

5 

0 

46 

134 

96 

71 

29 

0 

13 

38 

90 

8 

i ■ 

. 0 

103 

99 

78 

51 

75 

104 

22 

Q4 

0- 

_!2_ 

160 

12JL 

4 

0 

. 0 

l 

1 i 

1°W  0°  l°E 


PROBABILITY  OF  DETECT  ION  BY  THE  END  OF  TIME  6 » 


J 


0.435 


T_AB_L  E_  I_  V 1 £ _ ilL1  _?i? 

Myopic  Plan  — Hour  7 


TARGET  DISTRIBUTION 

NOTE*  ENTRIES  HAVE  BEEN  MULTIPLIED  BY  lo5 


1°N 


0 

70 

2 99 

482 

/43 

959 

679 

343 

50 

0 

150 

522 

700 

864 

635 

719 

749 

729 

540 

100 

325 

613 

744 

721 

709 

632 

643 

646 

754 

312 

704 

78  7 

656 

/ 2 7 

576 

533 

797 

762 

615 

640 

d 1 3 

618 

672 

516 

214 

241 

556 

720 

728 

771 

616 

705 

634 

"5T2~ 

214 

227 

THT 

605 

717 

B27 

747 

819 

706 

767 

502 

626 

749 

765 

783 

649 

344 

760 

69/ 

749 

596 

615 

774 

670 

697 

435 

00 

4 66 

695 

761 

/ 4 1 

612 

775 

751 

591 

108 

<L 

70 

4 10 

55/ 

769 

839 

75  l 

380 

f-22- 

0 

1°W 


1°E 


MYOPIC  ALLOCATION 


PROBABILITY  OF  DETECTION  BY  THE  END  OF  TIME  7 


0.492 


*f*«-*- 


9 


» 

I 


t 


j 


I 


IA5_le_^v-5 

RADIAL  FLEE  FROM  A UNIFORM  DISTRIBUTION 


Optimal  Plan  — Hour  4 

Note:  Shading  Indicates  the  cells  in  which  the  optimal  plan  places 
more  effort  than  the  myopic  plan. 
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Optimal  Plan  — Hour  5 

Note:  Shading  indicates  the  cells  in  which  the  optimal  plan  places 
more  effort  than  the  myopic  plan. 
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Optimal  Plan  — Hour  6 


Note:  Shading  Indicates  the  cells  In  which  the  optimal  plan  places 
more  effort  than  the  myopic  plan. 
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Optimal  Plan  — Hour  7 

Note:  Shading  Indicates  the  cells  In  which  the  optimal  plan  places 
more  effort  than  the  myopic  plan. 
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when  it  Is  normal.  Table  IV -6  showb  that  again  there  is  surprisingly  little  dif- 
ference between  the  detection  probabilities  for  the  myopic  and  optimal  plans. 

Example  3:  Two -scenario  target  motion.  In  this  example  the  target's  initial 
distribution  at  the  beginning  of  hour  1 is  circular  normal  with  standard  deviation 
10  miles  in  any  direction.  There  are  two  possible  scenarios  for  the  target's  motion; 
both  are  equally  likely.  For  scenario  1 the  target  is  assumed  to  be  traveling  at  a 
speed  uniformly  distributed  between  35  and  20  knots.  The  target's  course  is 
uniformly  distributed  between  150°and  210°.  The  target  maintains  a constant  course 
and  speed  chosen  from  these  distributions  until  hour  5 when  it  makes  an  independent 
draw  from  a distribution  on  speed  which  is  uniform  from  10-20  knots  and  an  in- 
dependent choice  of  course  from  a distribution  which  is  uniform  from  120-240°. 

The  target  continues  at  this  course  and  speed  for  the  remainder  of  the  problem. 

In  scenario  2 the  target  is  assumed  to  be  traveling  at  15  knots  and  to  make  a 
draw  from  a truncated  triangular  distribution  (see  Figure  II— 1)  with  mean  course 
75°,  maximum  course  105°,  best  course  90°,  and  weight  factor  2.  The  target  main- 
tains this  course  and  speed  until  time  5 when  it  makes  an  independent  draw  from  the 
same  distribution.  It  retains  this  course  and  speed  for  the  remainder  of  the  problem. 

As  in  the  first  two  examples,  we  have  4 hours  of  VP  search  available  beginning 
at  hour  4 and  we  wish  to  maximize  the  probability  of  detection  by  the  end  of  hour  7, 
The  detection  assumptions  are  similar  to  those  which  are  discussed  in  Example  1. 

I 

However,  the  sonar  conditions  vary  from  good  (W-200)  to  very  poor  (W~25)  In  the 

c 

> manner  shown  in  Figure  IV-l.  As  before  we  assume  that  there  are  12  ponobuoy 

i 

| hours  available  per  hour  for  each  time  period. 

i 

i 
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PR C3 ABILITY  OF  DETECTION  FOH  THE  OPTIMAL  AND  MYOPIC  PLANS 
FOR  RADIAL  FLEE  FROM  A UNIFORM  DISTRIBUTION 
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FIGURE  IV-1 


SONAR  REGIONS  FOR  EXAMPLE  3 


Tables  IV-7  and  IV-8  show  the  myopic  and  optimal  plana  for  this  example. 

At  hours  4 and  5 the  myopic  plan  concentrates  its  effort  on  scenario  2,  the  east- 
ward moving  one.  This  is  because  most  of  Its  probability  mass  is  located  in  areas 
of  relatively  good  sonar  conditions.  Even  though  by  hours  6 and  7 the  first  or 
south  moving  scenario  has  distinctly  the  higher  probability  given  failure  to  detect 
during  hours  4 and  5,  the  myopic  plan  contin>*^s  to  allocate  most  of  the  effort  to 
scenario  2 because  scenario  1 has  now  moved  into  the  very  poor  sonar  region.  By 
contrast  the  optimal  plan  concentrates  heavily  on  scenario  1 during  hours  4 and  5 be- 
fore that  scenario  moves  into  the  region  of  poor  sonar  conditions.  Then  during  hours  6 
and  7 the  optimal  plan  searches  scenario  2 which  has  moved  into  a better  sonar  region. 

Table  IV-9  shows  the  detection  probabilities  for  the  optimal  and  myopic  plan. 
Observe  that  at  hour  4 the  myopic  plan  has  a substantially  higher  detection  probability 
than  the  optimal  plan  but  that  by  hour  7 the  optimal  plan  has  detection  probability  . 58 
versus  . 48  for  the  myopic,  an  increase  of  19  percent  over  the  myopic  plan. 

Smoothing.  In  the  above  examples  a Monte  Carlo  simulation  of  10,  GOO  sample 
paths  was  used  to  represent  the  target  motion  process.  Even  with  this  rather  large 
number  of  replications,  ihe  target  location  distributions  still  have  considerable 
statistical  variation.  This  indicates  that  a smoothing  technique  such  as  the  one 
examined  in  reference  [s  ] might  be  helpful.  It  is  planned  to  investigate  this 


possibility  in  future  work. 
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Myopic  Plan  — Hour  5 
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Myopic  Plan  — Hour  7 
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Optimal  Plan  — Hoar  5 
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TABLE  IV-9 

PROBABILITY  OF  DETECTION  FOR  OPTIMAL  AND  MYOPIC  PLANS 
FOR  TWO  SCENARIO  EXAMPLE 


PROBABILITY  OF  DETECTION 


HOUR 

OPTIMAL 

MYOPIC 

1 

0.000 

0.000 

2 

0.000 

0.000 

3 

0.000 

0.000 

4 

0.116 

0.235 

5 

0.256 

0.374 

6 

0.506 

0.447 

7 

0.577 

0.486 

-115- 


We  suppose  that  we  are  given  N sample  paths  drawn  from  the  process  j 

{ Xt;  t = l,  ....  T} . The  n^1  sample  path  Is  assumed  to  have  the  form  x^),  w^,  \ 

where  xn  Is  the  position  of  the  target  at  time  s In  the  nth  sample  path  and  w Is  the  | 

S n 3 

til 

sample  probability  that  the  n sample  path  represents  the  target's  actual  path  (usually 

w = l/N). 
n 

Conceptually  the  first  act  of  the  optimizer  Is  to  convert  the  sample  paths  Into 

'll 

sequences  of  cell  numbers.  Thus,  the  n^1  sample  path  becomes  j 

01*  ^2 jT)f  Wn*  j 

where  jn  = c(x°)  Is  the  Index  of  the  cell  Into  which  x°  falls. 

8 S “ 

As  in  the  third  section  of  Chapter  11  we  define  the  function  £t  which  acts  on  a 
search  plan  ip  by  replacing  the  allocation  at  time  t,  i p(-  ,t),  with  f*.  an  optimal  allocation  1 

of  m(t)  effort  for  g (• , t),  the  posterior  target  distribution  at  time  t given  failure  to  detect 

" * \ 

at  all  times  other  than  t using  ip.  That  Is  j 

! 

<PU,b)  forest,  i 

H.  4>{i,  b)  - j 

f*(j)  for  s * t. 

j 

First  pass.  Set  ip j.t)  = 0 for  je  J,  t = 1,  . . . , T.  We  begin  with  t = 1.  For 

each  cell  J we  accumulate  all  the  probabilities,  w , of  paths  such  that  = j to  compute 

p (J),  the  probability  that  the  target  Is  in  cell  j at  time  1 for  j = 1,  ...»  J.  That  Is  1 

1 \ 

Pj  (J)  = 2 wn.  <IV-J| 

| 

Using  a standard  algorithm  (e.  g. , the  algorithm  In  Example  2.2.  8 of  reference  [hj,  j 

j 

j 

we  calculate  the  allocation  of  effort  f*  which  maximizes  the  probability  of  detection,  j 

i 

-116-  j 


(IV-2) 


iiPl(J)j^l-exp(-W(J)f*(J)/Ad))j  . 

for  a stationary  target  with  probability  p^O/  of  being  in  cell  j for  j = 1,  . . . , J, 
subject  to 


J 

^ f*0)  = m(t). 


(IV-3) 


Let  ^ <P q and  compute 

Wn  = Wn  exP^_w^i)  ^1^1*  for  n = 1,  ....  N.  (IV-4) 

The  value  wq  Is  the  probability  that  the  target  is  following  the  n&  sample  path  and  is 
not  detected  by  the  search  effort  at  time  1. 

For  t=  2,  we  compute  g^  ^fj»  2),  the  probability  that  the  target  is  In  cell  J at  time  2 
and  not  detected  by  the  search  applied  at  time  1 by 


tfy^(J’2) 


W_ 


for  J = 1,  ....  J. 


(IV- 5) 


so  that  It  sums  to  1.  As  above  we  find  an  optimal  allocation  of  m(2)  effort  for  the 
stationary  target  problem  with  distribution  g^(*  f 2),  and  let  ^2*^2  ^1*  Tbe  w®f8hta 
wQ  are  then  multiplied  by  exp(-W(j£)  ^2^2'  2)/A(J2))  to  the  revised  values 

of  wQ  for  n = 1 N. 

This  process  Is  continued  for  t=3 T.  At  the  end  of  this  first  pass  we  have 

calculated  the  myopic  plan,  1.  e. , the  plan  which  at  each  time  period  allocates  its 
effort  to  maximize  the  probability  of  detection  during  that  time  period  given  failuT 
to  detect  the  target  in  pievlous  time  periods. 


I 


Second  pass.  Let  w*  be  the  valu9  of  wft  obtained  by  the  end  of  the  first  pass 
described  above,  i.  e. , w*  is  the  probability  that  the  target  is  following  the  nto 
sample  path  and  Is  not  detected  by  the  effort  allocated  in  the  first  pass.  For  . 


we  calculate 


wq  = w*  exp^Wfl^)  1)/A(J^  for  n = 1,  ....  N, 

and  accumulate  w_  Into  cell  to  calculate  g..  (J,  1),  the  probability  that  the  target 

U 1 ▼’P 

is  in  cell  ] at  time  1 given  failure  to  detect  at  all  times  In  the  future.  Generally 

Vg*T<',1)* 

We  then  find  an  optimal  allocation  of  m(l)  effort  for  the  distribution  g^(* » !•)» 
set  = H ^T,and  compute  new  values  for  wQ  by 

wq  = wn  exp^-W(J°)  *T+1(j“  D/AflJ)^  . 


For  t = 2 we  calculate 


wn  = w^  exp  ^W(Jg)  ^ W>/Aa2))  * 


and  repeat  the  above  process  for  time  2.  This  continues  for  t=  3,  ,,,,  T and  the 
second  pass  is  completed.  Additional  passes  proceed  In  a similar  fashion. 
Observe  that  at  the  end  of  the  /***  pass 

l~PT[4>lT I = nJ1  V 


(TV-8) 


and  that  is  monotonlcally  decreasing  In  l.  We  shall  show  in  the 

third  section  that  as  l -<*>,  there  is  a convergent  subsequence  of  plans  { «ucb 

that  t)  allmj^  <PliT0,t)  exists  for  J = 1 J andt=l T and  such  that 
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ip*  Is  T -optimal.  Furthermore, 

Um b PTI^ixl  = PTW*1.  ffV-7) 

Thus  one  can  come  as  close  to  the  optimal  plan  as  he  wishes  by  making  enough 
passes.  For  computational  purposes,  one  usually  chooses  an  e > 0 and  stops  when 

pt(*Zt>-pt‘Vi)t1  < £' 

The  computer  program  which  Implements  this  algorithm  is  described  In  reference  [g]. 
Proof  of  Convergence 

In  this  section  we  take  the  point  of  view  that  a search  plan  ip  is  a density  defined 
on  Y * { 1,  . . . , T}  where  Y Is  the  plane.  Then  *(m)  becomes  the  class  of  plans  ip 
such  that 


fY  1>( y,t)dy  = m(t)  for  t = 1 T. 

The  search  plans  that  we  consider  In  this  chapter  are  a subclass  $(m)  of  *(m)  in 
which  the  effort  density  is  constant  over  the  grid  cells.  Thus  we  can  identify  a 
plan  ip  obtained  In  the  above  algorithm  with  a member  $ e ty(m)  as  follows:  Let 
c(y)  be  the  cell  containing  the  point  y.  Then 


$(M)  = toryeY,  t=l T. 


In  order  to  show  that  the  above  algorithm  converges  to  a plan  ip*  e $(m)  (under 
the  above  identification)  such  that 


P1  [ip*]  = max{  PT$] : # e $(m)}  , 
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we  first  find  necessary  and  sufficient  conditions  for  4>+  e ^(m)  to  be  T -optimal 

within  $(m),  i.  e. , for  to  satisfy  the  above  equality.  For  this  proof  we  shall 

begin  by  taking  the  point  of  view  that  there  is  a regular  detection  function  b (i.  e. , 

b has  a positive  continuous  and  strictly  decreasing  derivative  b')  such  that 
T 

b(Z^_1  ^pc^(«),t))  is  the  probability  of  detecting  the  target  given  it  follows  the 
path  u>.  Similarly 


PTW  - 


= Eb 0i«(3Ctl(w).tj|  . 


Let  F be  the  set  of  real-valued  Borel  functions  defined  on  Y * { 1, 
such  that 


T} 


Let 


/y  l*(y.  t)ldy  < ® for  t = 1 t, 

||f||  « h ess  sup  |fty,t)|  <®. 
t=l  y e Y 


, f(* , t)  is  constant  over  each  cell  in  the/ 

grid  for  time  t,  t = 1,  . . . , T j ’ 

{f  € F:f(y,t)  > 0 for  y c Y,  t = 1,  ....  T>  . 


In  Chapter  VI  we  show  that  PJ^.h],  the  Gateaux  differential  of  PT  at  $ in  the 
direction  h,  is  given  by 


pr'*'hl-t=x/Y 


Pt(y)  h(y,  t)dy 


for  ip  c F+,  h c K(^).  (TV-8) 


Let  p( j)  be  the  region  in  Y which  comprises  the  cell,  and  let 


6T».).t>  - Ip0)  v[b’(s?i  #(X»’*‘>)J  pt(y)dy 


for  j £ J,  t = 1,  . . . , T.  aV-9) 


Henceforth  we  shall  consider  all  f € £ to  be  functions  of  ] and  t for  J e J, 
t=l,  ....  T. 

For  (J>  e F+,  let  K{^)  be  the  cone  of  dlreotlons  h such  that  4>  + $h  c F+  for  all 
sufficiently  small  nonnegative  values  of  6.  For  tf>  c F+  and  h £ &($»),  (1V-8)  becomes 


PT(^*h)  = *i)?jD'r(4'’i,t)  h(J,tK 


(TV- 10) 


We  now  state  the  analogs  of  Theorems  1 and  2 of  Chapter  VI  for  the  class  of  plans 
£(m).  Let  S’j,  be  the  nonnegative  orthant  in  Euclidean  T space. 

THEOREM  1.  Suppose  b Is  concave  and  has  a bounded  nonnegative  derivative  b' . 
Then  il>*  is  T -optimal  within  $(m)  if  and  only  if  there  exists  (\(1),  ....  X(T))e<££ 
such  that 


ftT(^M,t)  = X(t)  if**0»t)>0, 


< X(t)  If  = 0,  for  jf  J,  t=  1 T. 


(IV- 11) 


Proof.  The  proofs  of  sufficiency  and  necessity  are  completely  parallel  to  the  ones 
given  for  the  proof  of  Theorem  1 of  Chapter  VI  provided  one  uses  J,t)  In  place 

of  DT(^*,y,t)  and  (TV- 10)  for  the  computation  of  Pjp. 

Let  Ejf  denote  expectation  conditioned  on  the  target  being  in  p (J)  at  time  t,  and 


define 


pt(J)  = / d)  Pt(y)dy  for  J £ J,  t = 1,  ....  T. 


Then  for  the  case  of  an  exponential  detection  function,  one  can  show  that 
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-WO)  ^o.t) 


« Ejt|oxp(-  W(XS)  *<Xa,s)jj  Pt0)  WO)e 

for  any  search  plan  ip  e £(m),  j * J,  and  t = 1 T.  As  above  we  think  of  the 

sweep  width  function  W and  the  allocation  functions  ip  e £ as  being  functions  of  (j . t) 
for  J f J,  t=  1,  . . . , T.  So  that  if  we  write  W(X  ) or  ip*(X  ,s)  we  understand 

o B 

this  to  mean  the  value  of  W or  ^*(* , s)  for  the  cell  in  which  Xg  falls. 

THEOREM  2.  Suppose  the  detection  function  Is  exponential  and  W 1b  bounded. 

Then  ip*  is  T -optimal  within  $(m),  if  and  only  if  there  exists  (X(l),  ....  X(T))c<§£ 
such  that  for  1 = 1.  . . . , J,  t = 1 T, 

EJt  exp^-  JtW0C8)  **(Xg,s)\  pt(J)  W(J)e~W(,)**0,t)  = X (t)  if  > 0 , 

L J (IV- 12) 

£ X(t)  if  = 0. 

Proof.  The  proof  follows  that  given  for  Theorem  2 of  Chapter  VI  using  the 
necessary  conditions  in  Corollary  2. 1.6  of  reference  [h  ). 

For  l = 1,  . . . , let  $ j be  the  member  of  *(m)  identified  with  ip^  in  the  manner 
discussed  at  the  beginning  of  this  section. 

THEOREM  3.  The  sequence  t 4>1T.  ) has  a convergent  subsequence 

A A 

and  the  limit  ip * of  this  convergent  subsequence  is  T -optimal  within  *(m)  for  the 
Monte  Carlo  sample  of  target  paths.  In  addition, 

Zli“®  Pt^/T1  = PT^*)-  (IV- 13) 

A 

Proof,  Since  each  satisfies 

fc  AO)  $/T0,t)  *m(t)  for  t = 1 T, 


-122- 


T,  J e J.  Hence  for  each  j and  t 


L»  • • • I 


it  follows  that  ^;T(J.t)  < m(t)/A(j)  for  t=  1, 
there  is  a sequence  { 1^}  ^ 2 such  that  lim^  ^/^Td»fc)  exists. 
Because  the  number  of  grid  cells  is  countable,  one  can  choose  a Cantor  diagonal 
sequence  to  obtain  a subsequence  { l.)  such  that  liim  w $>/  T0.t) 

exists  for  all ) and  t.  Let 

ba,t)  = lim  % (J,t). 

i -®  ‘iT 

Since  each  $ ^ T € $(m),  ty*  e *(m).  By  the  Lebesgue  dominated  convergence 
theorem  and  the  fact  that  - Pt^  Jor  ^ = 1»  • • • » equation  (IV- 13) 

must  hold. 

We  now  show  that  is  T -optimal  within  $(m).  For  the  nto  sample  path  define 

w;=»D“i>(-  JjWo^ 

Then  w*  is  the  probability  that  the  target  follows  the  ntt  sample  path  *nd  is  not  de- 

A 

tected  using  the  plan  Define 


£>U(J.t)  = zn  w*exp(W(J)^0.t)) 


for  t=  1 T,  J = 1,  .. . , T. 


Observe  that 


= EJtjexp^-  £ W(Xg)  0*(Xa,s)^  pt(J), 


provided  we  understand  { Xj,  t=  1,  . . . , T}  to  be  the  Monte  Carlo  sample  of  target 
paths.  Another  way  to  view  g^fl,  t)  is  as  a Monte  Carlo  estimate  of  the  expectation 
of  the  right-hand  side  of  the  above  equation  where  { Xg,  t = 1,  . . . , t}  is  the  under- 
lying motion  process  from  which  our  sample  paths  are  drawn.  We  shall  take  the 
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former  point  of  view  for  this  proof. 

Let  K(t)  = g^(J,  t)  and  g^(*  ,t)  = g^<* , t)/K(t).  We  claim  that  4>*  satisfies 

conditions  (IV- 12)  of  Theorem  2,  l.e. , there  is  a vector  (A(l),  ....  A(T))  of  non- 
negative numbers  such  that 


WO)  exp(-WO)  hQ,t))  = \<t)  if  > 0, 


(IV-14) 


< A(t)  if#* 0.t)  = 0,  j = l J,  t=l T. 


We  prove  this  claim  by  supposing  that  for  some  t,  condition  (IV-14)  does  not  hold. 
That  is  #*(< , t)  falls  to  satisfy  the  necessary  conditions  of  Corollary  2. 1. 6 of  reference 
[h  ) for  ty*(‘ , t)  to  maximize  probability  of  detection  for  cost  m(t)  for  the 
stationary  target  problem  with  target  probability  distribution  g£0(’ , t)  and 
exponential  detection  function  with  sweep  width  W(J)  for  the  cell.  Thus  one  can 
find  an  allocation  f*  such  that 

£ A(j)  f*(J)  * m(t), 

and 

^ *$*«»*>  j\-exp^-W(J)  f*0))j  > 

This  implies  that 

A J - J . 

1-PT(?'*|  = jSi  g^O.t)  exp(-WO)  ^*0.t))  > ^ g^O.t)  exp(-W(J)  f*(i)).  (IV- 16) 

Let  £ indicate  the  operation  which  maps  a plan  4’  Into  the  plan  £(^)  which  results 
from  performing  one  pass  of  the  algorithm.  Then  the  nondecreasing  nature  of  the 


■ * 
j=ig#*0,t)  J *exo(-W0)  ^*(J.t)) 
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detection  probability  resulting  from  each  of  the  stages  forming  a single  pass  com- 
bined with  (TV-15)  implies 


PT(S(i*)l  > PT$*}.  (IV- 19) 

Clearly  2 is  a continuous  operator  lr  the  sup  remum  norm  so  that 

Urn  2($ . T1  = S($*l. 

1 — ® ‘i1 

However,  since  P^  is  continuous  and  (TV- 13)  holds, 

Pt(2($*)J  = Urn  PT[B(JZ  T)1 
= lim  PT[^(ij+1)T] 

= PT[J*1. 

which  contradicts  (TV-15).  Thus  we  have  shown  that  satisfies  conditions  (IV- 12)  and 

A a 

by  Theorem  2,  >p*  is  T -optimal  within  *(m)  for  the  Monte  Carlo  sample  of  target  paths. 
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CHAPTER  V 

ALGORITHMS  FOR  OTHER  SEARCH  PROBLEMS 


In  this  chapter  we  outline  algorithms  which  can  be  used  to  solve  a number  of 
search  problems  related  to  the  main  one  considered  in  this  report.  These  algorithms 
are  based  on  necessary  and  sufficient  conditions  for  optimality  which  are  related  to 
the  ones  obtained  in  Chapter  IV. 

In  the  first  section  we  outline  an  algorithm  which  oan  be  used  to  calculate  plans 
that  maximize  probability  of  detecting  the  target  by  time  T when  the  detection  function 
is  not  exponential.  A method  of  calculating  plans  which  minimize  mean  time  to 
complete  a search  for  a moving  target  is  presented  in  the  second  section.  The  third 
section  outlines  an  algorithm  for  finding  optimal  survivor  search  plans.  The  final 
section  gives  a method  for  maximizing  probability  of  detection  when  there  is  no  constraint 
on  the  rate  at  which  effort  may  be  applied  but  only  on  the  total  effort  available  for  time 
1 to  T. 

None  of  the  algorithms  in  this  chapter  have  been  programmed  or  tested.  Thus, 
the  algorithms  should  be  viewed  as  approaches  to  solving  the  problems  and  not 
necessarily  as  answers. 

Optimal  Plans  for  Non-Exponential  Detection  Functions 

In  this  section,  we  consider  the  baste  search  problem  stated  in  the  first  section 
of  Chapter  I when  the  exponential  detection  function  is  replaced  by  a more  general 
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regular  detection  function  b.  A regular  detection  function  b:  [0,®)—{0, 1]  has  a positive, 
continuous, and  strictly  decreasing  derivative  b*.  As  in  the  third  section  of  Chapter  IV, 
we  consider  ^ to  be  a search  density  on  the  plane  so  that  the  probability  of  detection 
in  (1-3)  of  the  problem  statement  in  Chapter  I becomes 


PTW  = E 


As  in  Chapter  IV  we  shall  restrict  ourselves  to  allocations  which  are  constant 

A 

over  each  cell  of  our  search  grid.  That  is,  we  consider  only  plans  in  the  class  *(01) 
defined  in  Chapter  IV.  Recall  that  this  restriction  is  not  equivalent  to  assuming  that 
the  target  is  moving  in  discrete  space.  The  target  motion  process  X = {X  ,t=l,  . . . , T} 

l 

may  take  place  in  the  discrete  space  composed  of  the  grid  cells  or  it  may  take  place 
in  the  underlying  space,  the  plane,  over  which  the  grid  has  been  imposed.  In  either 
case 


p (J)  = Pr{X  is  in  cell  j}  for  j e J,  t = 1,  . . . , T. 


As  in  Chapter  IV,  let  E denote  expectation  conditioned  on  the  target  being  in 

J* 

cell  j at  time  T.  The  necessary  and  sufficient  conditions  of  Theorem  1 of  Chapter  VI 
for  a plan  i p*  e *(m)  to  be  T-optlmal  within  $(m)  become 


if  > 0 


i Mt)  if  < 0.  j € J,  t=  1 T, 


for  some  vector  (A(l),  . . . , A(T))  of  nonnegative  numbers. 
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Description  of  the  algorithm.  As  iu  Chapter  IV  we  assume  that  the  target 
motion  process  { X^,  t = 1,  . . . , T}  is  approximated  by  a large  but  finite  number  of 
sample  paths  drawn  from  this  process  in  a Monte  Carlo  fashion.  That  is,  we  suppose 
that  we  are  given  N sample  paths  drawn  from  the  process  {3^;  t=l,  ....  T}  . The 
sample  path  is  assumed  to  have  the  form 

. n n n. 

(Xj,  Xgi  •••»  XT),  WQ, 

where  x°  is  the  position  of  the  target  at  time  s in  the  0th  sample  path  and  w is  the 
s n 

sample  probability  that  the  nth  sample  path  represents  the  target's  actual  path 
(usually  = l/N). 

Conceptually  the  first  act  of  the  optimizer  iB  to  convert  the  sample  paths  into 
sequences  of  cell  numbers.  Thus  the  nth  sample  path  becomes 

..n  .n  .n 

dr  Jg*  " * * 3*p)> 

where  jn  is  the  index  of  the  cell  into  which  xD  falls, 
s s 

Let 


Then 


zq  • total  effort  density  that  accumulates  on  path  n 
T 

= Z 4>Qntt)  for  n = 1,  ...,  N. 
t=l 


Z 

n 


{n:jt=j} 


wb’(z). 
n d 


(V 


<y 
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The  algorithm  proceeds  as  follows: 

1.  Let  e > 0. 

2.  Let  be  an  Initial  allocation  and  compute  for  n *=  1,  . . . , N. 

3.  Set  l * 0. 

4.  Set  t * 1. 

5.  Find  the  allocation  f:  J-*(0,  « ] and  nonnegative  number  X (t)  such  that 
Zje  A(J)  = m<t)  for  J € J 

r{n;)f  = )}  wOb’(VWlfl'’,  + ,0))  ' X<t>  tt,m  > ° 

< x (t)  if  f(j)  = o. 

6.  Set 


^,T+t  ,(1.8)  forest 

W0,8)=  forj€j‘ 

f(j)  for  s = t 


7.  Set  t * t+1. 

8.  If  t < T,  go  to  step  5. 

9.  If  t > T,  set  l = 1+ 1 and  compute  d = | PT[^TJ  - PT^(/_1)X1 1 * 

10.  If  d < e stop;  ip  is  the  answer. 

11.  If  d > € , set  t = 1 and  go  to  step  5. 

The  difficult  part  of  this  algorithm  involves  finding  f and  X (t)  in  step  6.  One 
way  to  proceed  is  to  choose  a value  for  X (t)  and  then  solve  for  f(J)  for  j e J in  (V-6). 
Since  b’  is  monotone  decreasing,  this  is  a straightforward  numerical  problem.  One 
then  computes  2 jf(J)-  If  this  sum  is  larger  than  m(t),  then  one  should  lnorease  the 
value  of  X (t).  Correspondingly  if  the  sum  is  less  than  m(t),  X (t)  should  be  lowered. 
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Since  the  sum  is  a monotone  function  of  X (t),  a straightforward  numerical  aearoh 
procedure  can  be  used  to  find  the  value  of  X (t)  suoh  that  jf(J)  * m(t). 

The  disadvantage  of  the  above  algorithm  Is  that  it  requires  a much  larger 
number  of  calculations  than  the  algorithm  in  Chapter  IV  for  an  exponential  detection 
function. 

Minimizing  Mean  Time  to  Complete  a Search 

In  this  section  we  outline  an  algorithm  for  computing  plans  which  minimize 


mean  time  to  complete  a search  when  the  target  moves  according  to  a discrete 
time  and  space  finite  Markov  chain  and  the  detection  function  is  exponential  as  in 
Chapter  II. 

The  search  has  a cutoff  time  T so  that  the  search  will  proceed  until  the  target 
Is  detected  or  time  T Is  reached.  Thus,  if  the  target  is  not  found  by  the  end  of  time  - 

T,  the  search  will  stop.  We  say  the  search  Is  completed  If  either  the  target  Is  found 
or  the  search  has  been  stopped  after  time  T.  The  object  is  to  minimize  the  mean  time  ' 

to  complete  the  search.  ] 

j 

Let  4 be  the  mean  time  to  complete  the  search  using  plan  ip.  For  the  remainder  ! 

i 

of  this  chapter,  we  consider  ip  to  be  an  allocation  of  effort  as  described  In  Chapter  I.  Define! 

i 

.1 

P0[i/>]  = 0 and  let  PJ^J  be  the  probability  of  detection  by  time  t using  plan  ip.  Then  - 


E <1-P[*J) 
t-0 


= 1 + 


t r 
£ E exp 

t=i  L 


' t 

- £ W(X  )|(X  , s)/A(X  ) 
. 8=1  8 8 * 


fl 


Our  problem  Is  to  find  ip*  € *(m)  such  that 


* mln{u  T[^)  : >p  e *(m)}. 


Following  the  argument  In  Chapter  VI,  one  may  calculate  that  the  Gateaux  differential 


of  n at  ip  In  the  direction  h is 


„'I«.h)--r  z e 

U=1  j€  J 1 


“T 

ft  \ ' 

2 exp  1 
-t=u  ‘ 

-2  W(Xb)0(Xb,s)/A(X8) 

» s=l  / - 

,0)W(j)h0.u)/AC). 


Thus  a necessary  and  sufficient  condition  for  ip*  e +(m)  to  satisfy  (V-6)  is  that 
there  exist  nonnegative  numbers  \ (t)  for  t * 1,  ....  T such  that 


T 
2 E 


t=u 


Ju 


[«” (‘ jij"'*.****.- 8i/a<xb')]  pu®  Tm  ' 


\(u)  If  tp*(i,u)  > 0, 
<X(u)  if^(j,u)=0. 


For  J c J,  t = 1 T,  and  search  plans  ip,  define 


Ju 


(t.  4)  * EJu  j”exp  2 W(Xfl)^(X8.s)/A(Xs)^J  pu 


(j)  foru<t. 


s^u 


Equation  (V-7)  may  be  written  as 


(V— 7 ) 


!1 

] 

ij 

! 


T 

£ A (t,  ip*) 

ten,  JU 


W(l) 

A<J) 


exp  (-W(j)^  *0.  U)/A(J)) 


= X(u)  for  <!' *(J,u)  > 0, 
< A.  (u)  for  ip*{i,  u)  = 0. 


Define  R and  t as  in  the  third  section  of  Chapter  II.  Recall  that 


R(J>  t,  ip)  = Probability  the  target  reaches  cell  ] at  time  t and  is  not 

detected  by  the  effort  at  times  s = 1,  . . . , t-1  under  plan  ip. 


and  that  t ig  the  transition  function  for  the  process.  Assume  that  p gives  the 
initial  distribution  of  the  process  at  time  1 and  that  = 1 for  all  t > 1 and  j e J. 
Define 


V0»  t,  s,  ip)  = Probability  that  given  the  target  starts  in  cell  j at  time  t it  will  not 
be  detected  by  the  effort  at  times  u=t  + 1,  . . . , s,  under  plan  ip. 


It  follows  that  for  any  plan  j € J,  u = 1,  . . . , T,  and  t > u,  that 

A^(t,  ^ = R^*u*  ^ v0iU,t,  #)• 

We  now  describe  the  algorithm: 

(1)  (a)  Set  l - 0,  ^0(j.  t)  * 0 for  j e J,  t * 1 T, 

and  Hj.14>q]  =■  T. 

(b)  Choose  <•  > 0 and  compute  R<* , • , <l»0). 

(2)  Set  u * T. 

(3)  Compute 

Vt.V=R<,.u,V  V(j»u,t,  tpj  ) for  j e J,  t = u T. 

(4)  Find  f:  { 1,  . . . , J}-[0,®J  suoh  that  Z f(J)  = m(u)  and 

jeJ 


T 

s VV 

tai  * 


WQ)  -W<j)fO)/AU) 
AO) 


= \(u) 

< M“) 


for  f0)  > 0, 
for  f0)  * 0. 


(5)  Set 


4>l  0»s)  fors/u, 
fO)  for  s = u. 


(6)  If  u * 1,  set  l * /+1  and  go  to  step  (9). 

(7)  Compute 

V0,u-l,t,  ) * 2 Tu-i^,k)  exp  ( _W(k)  */+10t»u)/AO)J  V(k,u,t,  4>t) 

‘ ke  J ' * 

for  je  J,  t = u T. 
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•flg^rifyTTnginr  a a « rr> 


'.JT.'Wi 


1 1 
t 


! 

i 

(8)  Set  u = u-1,  l = l+l  and  go  to  step  (3). 

T 

(9)  Compute  R(-f  • , 4>  ) and  M [if>  ] = 1 ♦ £ £ R(j,  t,  H>  ) t)/A(J). 

! t*l  j(J 

i 

(10)  Compute  d = | m Tt^zl  '»*Tl*j..Tll  • 

(11)  If  d > € , go  to  step  (2). 

<12)  If  d < e,  stop;  tj>  is  the  answer. 

Note  that  step  (4)  is  equivalent  to  finding  the  optimal  allocation  of  m(t)  effort 
for  a density  function  d given  by 

T 

d(j)  = £ a (t,  * ) for  j e J. 
t=u  ' 

The  algorithm  in  Example  2.  2.  8 of  reference  (hi  will  find  such  an  f*. 

Optimal  Allocation  for  the  Survivor  Search  Problem 

In  this  section  we  consider  a discrete  time  version  of  the  survivor  search 
problem  addressed  in  reference  (t  ].  The  target  is  stationary  with  distribution  function 
p where 

P(J)  = Pr{ target  in  cell  j}  forjeJ. 


The  target  has  a stochastic  lifetime  whose  distribution  may  depend  on  the  target's 
location.  In  particular,  we  assume  that 


<p(j,  t)  = Pr { target  dies  at  the  end  of  time  1 1 target  is  in  cell  j} 

for  j c J,  t = 0,  1,  ...  . 
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I wi  jpKTE 


Now,  we  can  write  (V— 10)  as 


£illM 

A(j) 


T 

Z (pQ.t)  U(J,t,  1>*) 


t-8 


= X(S) 

< X(8) 


if  ^*0.8)  > 0, 
if  4>*0.8)  = 0. 


For  any  plan  4 , define 


Ufl0.t.  4)  = exp 


u^a 


= UO.t,  4>)  exp 


for  jc  J,  s < t < T. 


Then  conditions  (V-ll)  can  be  written  as 


A0) 


T 

Z 

t*=s 


uso.t,**) 


exp 


(-■Tof 


= X(8)  if^*0.8)>0, 

<x(e)  if  <ro.8)  *o. 


The  following  gives  a description  of  an  algorithm  for  calculating  optimal  survivor 
search  plans: 

(1)  Choose  e > 0 and  set  l = 0. 

(2)  Set  (J.t)  = 0 for  J 6 J,  t = 1,  . . . , T. 

(3)  Set  UO.t,  40)  * 1 for  J e J,  t = 1 T. 

(4)  Set  s = T. 

*5)  Compute 

* / wm  \ 


U (j,t,  4.)  ~ U0,t ,4,)  exp 


for  J c I,  t = s,  . . . , T. 


I 

i 


(6)  Solve  for  f : J— [0,®)  such  that  2^  jf(j)  = m(t)  and 


-ElMi-T  2 «>(j,t)  U(J,t,  ^)1  exp  = X(s)  if  f<j)  > 0, 

t=8  ' ' < X (s)  if  f(J)  = 0. 


(7)  Set 


for  t / s 

^z+1(j.t)  = for  j € J. 

f(j)  for  t = s 


(8)  If  s = 1,  go  to  step  (11),  otherwise  got  to  step  (9). 


(9)  Compute 

U(J,  t,  <Pl+l)  - Ug(j.  t,  <Pt)  exp  for  )€  J,  t ■=  8 T. 

(10)  Set  s - 8-1,  l *1+1,  and  go  to  step  (5). 

(11)  Set  l *1+1  and  compute  d = | sTt^^J  ~ I • 

(12)  If  d > c go  to  step  (4). 

(13)  If  d _<  e stop;  ip  is  the  answer. 

Observe  that  2.  p(j)  f 2^  «?(j,  t)  U(J.  t,  ip) 1 =2^  2.  T p(j)  c^(J,t)U(J,t,  ip) 

Jt  d L t— S J t~8  JC  d 

is  the  probability  that  the  target  dies  In  the  interval  (s,  T]  before  being  detected  by 
plan  ip.  Step  (6)  of  the  algorithm  finds  the  allocation  of  effort  for  time  8 which  minimizes 
this  probability  given  that  the  allocation  at  all  times  other  than  8 is  already  specified. 
Observe  that  for  any  plan  ip 


l-sm=2  2 p(J)  tfd.t)  U(J,t,0). 
t=0  Je  J 


.ij.  i-. 


The  optimization  procedure  first  reallocates  effort  at  time  T to  minimize  the  t * T 
term  in  tne  above  sum.  Then  it  reallocates  the  effort  at  T-l  to  minimize  the  sum 
of  the  t = T-l  and  T terms.  For  time  s it  reallocates  the  effort  for  that  time  to 
minimize  the  sura  ofthet  = stot  = T terms  in  (V-12).  Thus,  sTl^+1l  > sTl^jl  for 
/ = 0,  1,  ...  . 

Notice  that  finding  the  ralutlon  f in  step  (6)  is  equivalent  to  finding  the  allocation 
of  m(s)  effort  which  maximizes  the  probability  of  detecting  a stationary  target  with 
defective  location  distribution  2 given  by 

T 

<5(1)  = P(l)  ~ <P(M)  U(Jit.  ^.)  forjej, 
t=s 

and  exponential  detection  function.  This  allocation  may  be  performed  by  the  algorithm 
given  in  Example  2. 2. 8 of  reference  [h). 

One  can  show  that  the  above  algorithm  converges  by  using  an  argument  similar 
to  the  one  given  in  the  third  section  of  Chapter  in. 

Allocating  the  Total  Effort  Available 

For  the  other  search  problems  considered  in  this  report  we  have  assumed  that 

search  effort  becomes  available  at  the  rate  m(t)  for  t = 1 T.  By  contrast,  in 

this  section  wc  assume  that  there  is  a total  amount  M of  effort  which  can  be  applied  at 
any  rate  the  planner  wishes.  With  the  exception  of  having  the  constraint  on  total  effort 
rather  than  on  the  rate  at  which  effort  can  be  applied,  wc  return  to  the  basic  search 
problem  described  in  Chapter  I. 

This  type  of  problem  occurs  when  there  is  only  one  aircraft  flight  available  to 
search  for  a submarine  in  some  interval  t = 1,  . . . , T of  time  and  the  search  planner 
wishes  to  select  the  best  time  to  search.  Solving  this  problem  is  equivalent  to  solving  T 
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stationary  target  problems.  One  simply  computes  the  target's  distribution  at  time  t 
given  that  no  search  has  been  applied  and  finds  the  optimal  allocation  of  M units  of 
effort  to  that  distribution  for  t = 1,  . . . , T.  The  planner  then  chooses  the  time  t* 
which  yields  the  highest  probability  of  detection  and  follows  the  optimal  allocation  for 
that  time. 


A more  interesting  problem  arises  when  one  is  not  required  to  allocate  all  of 


his  effort  during  a single  time  period.  In  this  case  one  can  show  that  a necessary 

T 

and  sufficient  condition  for  a plan  ip*,  such  that  E.  Z 0*(j,  t)  = M,  to  maximize 

je  J t— l 

the  probability  of  detection  by  time  T within  the  class  of  plans  which  allocate  M units 
of  effort  is  the  existence  of  a A.  > 0 such  that  for  je  J,  t = 1,  ...»  T 


: [exp  (-  Z tt*(X 

J‘1;  V A0>  s* 


, 3)) J pt0 


0>  ■7ofe!tp (■'!«'  **0,t))  = ^ If  f’O.t)  >»,  (V-13 

< A if  0*0, t)  = 0. 


For  a search  plan  0 let 


,a,t)  = E.t  [exp  (- 


,3))] 


Pt(j)  for  t = 1 T,  and  j e J. 


The  following  is  a proposed  algorithm  for  solving  the  above  problem: 

(1)  Make  an  initial  guess  ip  for  the  search  plan  and  choose  e e > 0. 

(2)  Choose  A > 0 and  set  l = 0 and  k = 0. 

(3)  For  t = 1,  . . . , T do  steps  (4)  - (6). 

(4)  Compute  g . (• , t)  and  solve  the  following  equation  for  f(j): 


a,t)  "fyfexp 


Tyf  f°>)  = for  > £ J- 


best  WMLABLE  COPT 
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(5)  Let  f*(J)  * max{  0,  f(j)}  for  j c J and  set 


*l+ i«’8> 


ipfl.s)  for  aft 
f+(J)  for  e=t. 


(6)  Set  1 = 1 + 1. 

(7)  If  I > 0,  and  max  { Uj  (j,t)-^_T(J,t)|  } < c x go  to  step  (9). 

je  J,  t=l T 4 

(8)  Otherwise  go  to  step  (3). 

T 

(9)  Calculate  Cltfi.]  =2  2 ^.(J.t). 

jeJ  t=l 

(10)  If  | C[4>i  ] - M | < e2>  stop;  \i>l  is  the  answer. 

(11)  Otherwise  set  k = k+1  and  choose  X^  to  be  larger  than  X^  If 

C(^)  > M and  to  be  smaller  than  X^  if  C[i^]  < M. 


In  steps  (3)  - (8)  the  algorithm  Is  recursively  calculating  an  allocation  4>  which 
satisfies  (V-13)  for  X = X^.  Once  this  is  done,  at  least  approximately,  the  algorithm 
checks  the  total  effort  C(^]  associated  with  this  allocation.  If  the  total  effort  is  too 
large  (small),  then  X^+1>  the  next  guess  for  X,  1b  made  larger  (smaller)  than  X^. 
Because  C[^>]  is  a monotone  decreasing  function  of  X , this  will  cause  the  CI^J  resulting 
from  X to  be  smaller  (larger)  than  that  obtained  from  X . Thus  steps  (9)  - (11) 

k + 1 K 

should  really  be  thought  of  as  performing  a binary  search  to  obtain  a value  of  X such 
that  C(^)  = M. 
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CHAPTER  VI 


GENERAL  NECESSARY  AND  SUFFICIENT  CONDITIONS 
FOR  MOVING  TARGET  PROBLEMS 


In  this  chapter*  we  find  necessary  and  sufficient  conditions  for  a search  plan 
to  maximize  the  probability  of  detecting  a moving  target  by  time  T under  constraints 
on  the  rate  at  which  search  effort  may  be  applied.  These  conditions  apply  to  a very 
wide  variety  of  moving  target  problems  in  continuous  or  discrete  time  and  contlnous 
or  discrete  space.  Many  previous  results  concerning  necessary  and  sufficient  conditions 
for  moving  target  problems  appear  as  special  cases  of  the  results  obtained  here.  In 
particular  our  results  include  the  necessary  conditions  obtained  by  Heilman,  reference  [u], 
for  diffusion  processes  and  by  Saretsalo,  reference  [v],  for  continuous  time  and  space 
Markov  processes,  and  they  extend  those  results  by  showing  that  the  conditions  are  also 
sufficient.  The  results  of  Stone,  reference  (w],  and  Persiheimo,  reference  [xj,  for 
continuous  time  generalized  conditionally  deterministic  motion  are  special  cases  of  the 
results  in  this  paper  as  well  as  those  of  Brown,  reference  (b],  for  discrete  time  and 
space  target  motion. 

The  results  proved  in  this  chapter  are  not  restricted  to  problems  In  which  targets 
move  according  to  a Markov  process  or  a mixture  of  Markov  processes.  The  results 
apply  to  any  process  for  which  the  expectation  defining  the  function  in  (VI-4)  makes 
sense.  In  the  case  where  the  detection  function  is  exponential,  calculation  of  DT  Is 
equivalent  to  being  able  to  calculate,  for  each  time  t and  point  y in  the  search  space, 

* This  chapter  is  based  on  reference  [ej* 
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the  probability  (density)  that  the  target  passes  through  point  y at  time  t and  is  not 
detected  by  the  search  effort  over  time  [0,  T). 

In  the  case  of  a discrete  time  target  motion  process  and  an  exponential  detection 
function,  the  necessary  and  sufficient  conditions  have  the  simple  intuitive  Interpretation 
given  in  the  basic  condition  of  Chapter  I.  This  condition  forms  the  basis  for  the 
algorithms  in  Chapters  II  and  IV. 

In  the  first  section  of  this  chapter  we  state  the  generalized  version  of  the  optimal 
search  problem  of  Chapter  I that  we  are  considering  here.  The  statement  of  the 
problem  and  the  theorems  are  given  in  terms  of  a discrete  time  and  continuous  space 
target  motion  model.  Modifications  required  to  apply  the  results  to  discrete  space  or 
continuous  time  are  noted  after  the  results  are  stated.  A discrete  time  model  is  used 
because  it  1b  most  amenable  to  numerical  calculation  and  because  discrete  time  allows 
us  to  present  proofs  that  are  considerably  simpler  than  those  required  for  continuous 
time.  A continuous  space  is  chosen  for  the  basic  presentation  because  it  illustrates  that 
the  results  are  not  simply  applications  of  the  Kuhn-Tucker  theorem.  In  addition,  the 
discrete-space  results  are  usually  transparent  once  the  continuous-space  results  have 
been  obtained. 

In  Theorem  1 of  the  second  section  we  prove  that  conditions  (IV -6),  stated  below, 
are  necessary  and  sufficient  for  the  optimality  of  a search  plan  in  discrete  time  and 
continuous  space  when  the  detection  function  is  ooncave.  In  the  case  of  continuous 
time,  we  observe  that  the  necessary  conditions  (VI-6)  are  true  but  their  proof  is  not  a 
simple  extension  of  the  one  given  in  Theorem  1.  Instead  the  reader  is  referred  to  the 
proof  of  Theorem  5. 2 of  reference  (w]  which  may  easily  be  applied  to  proving  this  result. 
However,  the  proof  in  reference  (w)  is  very  technical  involving  a demonstration  of  the 
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existence  of  a measurable  selection  from  a function  Bpace.  ThlB  difficulty  la  cot 
present  in  discrete  time.  In  Theorem  1'  of  the  second  section  we  present  a unified 
statement  of  the  necessary  and  sufficient  conditions  which  applies  to  a target  motion 
process  with  any  combination  of  discrete  or  continuous  space  and  time. 

The  third  section  considers  the  special  case  of  a discrete  time  target  motion 
process  and  an  exponential  detection  function.  For  this  case  we  give  a simple  proof  of 
the  necessity  result  which  relies  only  on  the  necessary  conditions  for  an  optimal  search 
plan  for  a stationary  target.  In  this  case,  the  necessary  and  sufficient  conditions  have 
the  simple  Intuitive  interpretation  given  in  the  basic  condition  of  Chapter  I;  namely, 
at  each  time  t,  the  optimal  plan  allocates  its  effort  for  that  time  period  so  as  to 
maximize  the  probability  of  detecting  a stationary  target  with  location  distribution  equal 
to  the  posterior  target  location  distribution  given  failure  to  detect  at  all  times  other 
than  t,  i.  e. , at  all  times  before  and  after  time  t.  For  discrete  time  and  Bpace  target 
motion,  Washburn,  reference  |y ],  has  shown  that  the  basic  condition  is  necessary  but 
not  sufficient  when  the  searcher  obtains  Independent  glimpses  at  the  target  at  each  time 
period,  but  the  detection  function  during  a single  time  period  is  not  exponential. 

In  order  to  avoid  confusion,  the  reader  should  note  that  we  have  adopted  a 
different  detection  model  from  that  used  by  Washburn.  In  the  discussion  in  the  first 
section  below,  the  reader  will  see  that  we  have  assumed  a constant  detection  function  b 
over  all  time  periods  and  that  this  function  relates  the  total  effort  which  falls  on  the 
target  during  the  search  to  the  probability  of  detecting  the  target.  Only  when  b Is  an 
exponential  detectivu  function  does  one  have  Independent  glimpses  at  the  target  at  each 
time  period.  Thus,  the  model  assumed  by  Washburn  coincides  with  the  one  used  In  this 
report  only  when  the  detection  function  is  exponential. 


Problem  Statement 

Let*{xt;  t = 0,  1,  . . . , T}  be  a discrete  time  stochastic  process  where 

takes  values  in  Euclidean  n-space,  Y,  and  T is  a positive  integer.  The  random 

variable  represents  the  target's  position  at  time  t.  We  assume  that  (X^,  X^) 

T+l 

has  a joint  density  function  p defined  on  Y . Let  pt  be  the  density  of  the  marginal 

distribution  of  X^  for  t * 0,  1 T. 

Let  4>(‘ , t)  be  the  allocation  of  search  density  at  time  t for  t * 0,  1,  . . . , T. 

That  is.  tp(y,  t)  is  the  effort  density  applied  to  point  y at  time  t under  plan  tp.  We 
assume  that  >P  is  a member  of  the  space  F of  real-valued  Borel  functions  f defined 
on  Y x { 0,  1,  ....  T}  such  that 


/ Y I f(y.  *)  I dy  < » for  t = 0,  1,  ....  T, 

T 

||f||  a 2 ess  sup  j f (y,  t)|  < ®. 
t*0 

Then  F is  a linear  space  with  norm  1 1 f 1 1 for  f e F.  Let  F+  * { fe  F : f > 0}  , and  let 
m(t)  > 0 be  the  amount  of  effort  which  is  available  at  time  t for  t = 0,  1,  . . . , T.  Define 


♦ (m)  = {tpe  F+:  / Y^(y,  t)dy  = m(t)  for  t • 0,  1,  2 T} 


(VI-1) 


Let  X(o.  * ) denote  a sample  path  of  the  process  X.  If  the  target  follows  this 
path  and  we  allocate  search  effort  according  to  the  plan  tp,  then 


bl  2 tp(X(fjj,B),B) 
\s=0  j 


is  the  probability  of  detecting  the  target  by  time  T.  The  function  b is  called  a detection 
function.  It  relates  the  accumulated  search  density  along  the  target's  path  to  probability 


* In  distinction  to  previous  chapters,  we  take  the  initial  time  to  be  0.  This  is  to 
facilitate  a combined  statement  of  continuous  and  dlsorete  time  results  in  Theorem  1'. 


N 


I 

] 

A* 


of  detection.  Letting  E Indicate  expectation  over  sample  paths,  we  have  that  the 
overall  probability  of  detection  by  time  T Is 


1 


.8).  8)  j j. 


pTm  E E bl  z 

' 8=0 


In  the  sequel  we  shall  usually  not  indicate  the  dependence  of  X on  u. 

The  optimal  search  problem  under  consideration  is  to  find  ip*t  *(m)  such  that 


P.pl!/'*!  = max{ *(m)}  . 


A plan  ip*t'J/(m)  that  satisfies  (VI-2)  is  called  T -optimal  within  ■Km). 


I 

1 


Necessary  and  Sufficient  Conditions 

In  this  section  we  find  necessary  and  sufficient  conditions  for  a plan  ip  * to  be 
T -optimal  within  'l'(m)  when  the  detection  function  b is  concave.  The  conditions  Involve 
the  Gateaux  differential  of  P^,  which  we  will  now  define  and  calculate  under  the 
assumption  that  for  some  finite  k > 0,  the  derivative  b’  of  b satisfies  0 < b'(z)  < k 
for  z >0. 

Gateaux  differential  of  P^.  Let  ip,  he  F.  If 


PUiMil  5 lim  f (p  (^eh)  - P [<|>] 
e-0  ' 

exists,  then  P^l^,  h)  is  the  Gateaux  differential  of  P^  at  ip  in  the  direction  b. 

4*  -f 

For  ip€  F , let  C(^)  be  the  cone  of  directions  h such  that  ^ + ffh  e F for  all 
sufficiently  small  nonnegative  values  of  0.  Now  for  i pe  F+  and  he  C (ip), 


PL  [(Mi]  - lira 

e-0 


,8=0 


1p(Xg,  S)  + €h(Xfl,8) 


<P(Xs,  8) 
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Since  the  integrad  Is  bounded  by  « 1 1 h 1 1 , we  may  invoke  the  dominated  convergence 
theorem  to  obtain 


PJ^.h]  - E 


b’  Z *(X 


8=0 


\ T 

,8)  z h(X,t) 
/ 1=0 


Z b'(  Z $ (X  , a)^  h(X.,t) 
t=0  \ 8=0  / 


Let  pt  denote  the  marginal  density  for  X^,  and  let  E^  denote  expectation  conditioned 


on  = y.  Then  for  ip  € F and  he  C(^). 


PT1<,,h|  ' £/*■»« 


Note  that  we  have  expressed  P^[^,  * j as  a linear  functional  on  C(tp).  Define 


Dx(*.y.t>  = Eyt 


b'^  Z ip(Xgt  s)^j  pt(y)  for  if,  e F+, 


ye  Y,  t = 0.  1 T. 


(VI-3)f 


/ T \ 

bM  Z ^(X  , s) 

L \ s=o  s / -I 

b(y,  t)  pt(y)dy. 

(VI-4) 


Then 

T 

= Z J yD-W.y.t)  h(y,t)dy  forheC(^).  (VI-5) 

t=0 

Necessary  and  sufficient  conditions.  We  now  Btate  and  prove  necessary  and 
sufficient  conditions  for  T-optimality.  Let  <^T+1  be  Euclidean  T+l-space  and  <^+ 
be  the  nonnegative  orthant  in  . 

THEOREM  1.  Suppose  that  b is  concave  and  that  it  has  a bounded  nonnegatlve 
derivative,  b1.  Then  >l>*  is  T-optlmal  within  *(m)  if  and  only  if  there  exist 
(MO) MT))  e * | such  that 
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nT(v  Vv,t)  - \(t)  if  Ip  + (y.  t)  .>  0, 

< X (t)  if  4>*( v.  t)  - 0 for  a.  e.  ye  Y,  t = 0,  1,  ....  T. 

Proof  of  sufficiency.  The  proof  of  sufficiency  follows  that  of  Theorem  8.4. 1 in 
reference  [h].  The  essence  of  this  argument  is  due  to  D.  H.  Wagner. 

Observe  that  the  concavity  of  b implies  that  P is  a concave  functional  on  F+. 
We  now  proceed  to  use  an  argument  by  contradiction.  Suppose  that  ip*e^(m)  satisfies 
(VI-6)  and  that  there  is  a ^e^(m)  such  that  P^[i/j]  > P,p[#*]. 

Since  P^  is  concave  we  have  for  0 < 6 < 1, 

p T[0*  + 0(4>-4>*)l  - ptI^*1  = PT[(i-ew*  + on  - pt[^*] 

> <i-e>PT[**]  + 0PT[*]-PT[**] 

= 0(PT[^]  - PT[**J). 

It  follows  that 

Py**,*-**]  >PT[^]  - PT[V  *]  > o. 

However,  by  equations  (VI-5)  and  (VI-6)  we  have 
T 

f D (ip*, y,  t)[ip(y,t)-ip*(y,t)]dy 
t-0 

f 

1 Uy:4>*(y,t)>0}DT(lp*,y’t)[!p(y’t)-!f,*(y’t)]dy 

T 

' t“  ^{y:^*(y>t)-0}UT(^+’y*t)[^(y>t)"^*(y-t)]dy 

T 

1 - / ,.X  (t)[ </> (y , t)-</'*(y,  t)]dy  = 0, 


where  the  last  equality  follows  from  the  fact  that 


/y  4>(y>  t)dy  * /y  ip*(y,t)dy  = m(t)  for  t = 0 T. 


However  (VI-8)  contradicts  (VI-7)  and  sufficiency  is  proved. 

Proof  of  necessity.  Suppose  ip*  is  optimal  within  *(m).  Since  b is  increasing, 
we  observe  that  tp*  is  also  optimal  within  the  larger  class  obtained  by  replacing  the 
equality  constraint  by  an  inequality  constraint  in  the  definition  of  ♦(m)  in  (VI-1). 

Since  / f(y,  t)dy  < ® for  fe  F*  and  t = 0,  ....  T,  we  may  define 
T 

L [f]  = P [f]  - S y(t)  [/  f(y,t)dy  - m(t)l  for  fe  F+,  y.e  <?  . 

' t=o  L J 1 

By  Theorem  1 on  pa&e  217  of  reference  (zj,  there  exists  \e  such  that 

For  fe  F+  and  he  C(f),  the  Gateaux  differential  LJJf,  hj  of  at  f in  the  direction  of 

h exists  and  is  a linear  functional  of  h.  In  particular  by  (VI-5) 


Ljjf.h] 


T T 

Z fyDr(f>y,t)  h(y,t)dy  - s X (t)  /^i(y,  t)dy 
t=0  A t=0  1 


(VI 


Following  page 227  of  reference  (zj,  we  observe  that  since  L [^*]  = max 

A Ip  € 

we  have  for  any  >pc  F+, 


lira  7{L.[f*+«(H,))-L.[(*)}  < 0. 


€ — 0 


Thus 


ip-ip*]  < 0 for  tpe  F*. 


(VI 
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Sotting  v - L;'  *,  we  obtain 


0>  ( 0 *,  ~h4‘  *j 


■2  L 'x[4'\4>*h 


while  setting  4 =24'*  yields 


Vx[ip*,4’*]  <o. 


Equations  (VI-9),  (VI-11),  and  (VI-12)  imply 
T 

2 /Y[D  (**,y,t)  -X(t)J  ip*( y,t)dy  = 0, 
t=0 

while  equations  (VI-10)  and  (VI-13)  imply 
T 

2 / [D  (ip*,  y.t)  - X(t)J  ^(y.t)dy  < 0 for  all  L . . 
t=0 

Equation  (VI-14)  implies  that  *,  y,  t)  < X (t)  for  a,  e.  yeY,  t = 0,  1,  T,  and 

(VI-13)  implies  D^(i/j  *,  y,  t)  = X (t)  for  a.  e.  yeYandt  = 0,  1 T such  that 

ii'*(y,  t)  > 0.  The  necessity  of  the  conditions  in  (VI-6)  follows,  and  the  theorem  is  proved. 
In  the  following  paragraphs  we  discuss  extensions  and  specializations  of  Theorem  1. 
Discrete  space  - discrete  time.  Theorem  1 and  the  proof  given  above  also  hold 
when  the  search  space  Y is  discrete  provided  one  interprets  p^(y)  in  (VI-4)  as  the 
probability  that  - y. 

Continuous  time  (sufficiency).  The  obvious  analog  of  the  sufficiency  part  of 

Theorem  1 holds  for  continuous  time  provided  the  target  motion  process  {X^;  0 <■  t < T} 

T 

has  Morel  measurable  sample  paths,  the  conditional  expectation  "FI  [b’(/  Q 4 (Xg,  s)ds]  is 
well  defined,  and  we  take 


(VI-ff]| 


(VI-12] 


(VI-131 


3 


(VI-14* 


l>  i 

' I 


vl 


„f,.T 


\ j ()  4>(X^,  s)ds 


Pt(y)  for  . yt  Y,  te  [0,  TJ. 


IS 


(VI -15)1 
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The  proof  of  this  assertion  parallels  the  sufticiency  proof  in  Theorem  1.  The  Borel 

T 

measurability  of  the  sample  paths  is  needed  to  guarantee  that  the  integral  / ^(X  , s)ds 

v S 

is  veil  defined. 

Continuous  time -continuous  space  (necessity).  The  corresponding  necessity 

result  for  continuous  time  is  more  difficult  to  prove;  its  proof  is  not  an  obvious 

extension  of  the  one  given  for  Theorem  1.  However,  by  paralleling  the  proof  of 

Theorem  5. 2 in  reference  (wj  one  may  show  that  th*-  onditlons  in  (VI-6)  are  necessary 

with  defined  as  in  (VI-3  5).  In  fact,  that  proof  snows  that  for  continuous  space  the 

necessary  result  holds  when  the  concavity  assumption  on  b is  dropped.  The  proof  in 

reference  [wj  also  allows  one  to  add  a constraint  of  the  form  0 < 4>(y,  t)  < B for  some 

positive  number  B with  a corresponding  change  in  the  conditions  in  (VI-6). 

-2 

Observe  that  when  b(z)  * 1 - e , we  have,  fbr  continuous  time, 


DT(d,y,t)  « 


T 

0 


4(Xs,s)ds 


Pt<y>. 


(VI-16) 


If  {X  ; s > 0}  is  a Markov  prooess,  then  one  can  show  that 
8 


DT(V',y,  t)  = Jx  r(x,  0,y,  t,  it)  R(y,t,  T,  ip)  pfl(x)dx,  (VI-17) 

where  under  plan  <t,  r(x,  6,  yt,  it)  is  the  probability  density  that  at  time  t the  target  is 
located  at  y and  is  undetected  given  it  was  at  x at  time  0,  and  R(y,  t,  T,  it)  is  the 
probability  that  if  the  target  ie  at  point  y at  time  t It  will  be  undetected  in  the  interval  (t,  TJ, 
Because  of  **ie  Markov  nature  of  the  process,  r(x,0,y,t,  it)  R(y,  t,  T,  it)  ta  the  probability 
density  that  a target  starting  at  x at  time  0 will  pass  through  the  point  y at  time  t and 
remain  undetected  throughout  [U,T|.  The  integral  on  the  right  of  tVI-17)  averages  over 
the  distribution  of  the  target’s  petition  at  time  0 to  obtain  the  probability  density  that 


-lftn- 


the  target  passes  through  point  y at  time  t and  remains  undetected  throughout  [0,  T].  Now 


exp 


K 


4>(X.(o),  s),  s)ds 


is  the  probability  of  failing  to  detect  the  target  by  time  T given  it  follows  path  go.  Thus, 
the  right-hand  side  of  (VI-16)  is  simply  the  probability  density  of  the  target  passing 
through  point  y at  time  t and  failing  to  be  detected  by  time  T.  From  this  observation 
and  the  above  discussion,  equation  (VI-17)  follows. 

Thus,  Saretsalo's  Theorem  5. 1 (reference  [v])  and  the  theorem  of  Heilman 
(reference  [u])  are  special  cases  of  the  necessity  result  obtained  in  this  chapter,  m 
fact,  the  specialization  to  Markov  processes  given  here  is  stronger  than  the  result  in 
reference  [v]  in  the  sense  that  no  assumptions  concerning  the  continuity  of  the  transition 
function  are  required.  In  addition,  we  have  proved  sufficiency. 

It  also  follows  that  the  necessary  and  sufficient  conditions  obtained  by  Stone  in 
reference  fcv]  and  Persihelmo  in  reference  frt  ] are  a special  case  of  the  conditions  found 
in  this  chapter.  In  particular,  one  can  show,  in  the  notation  of  Stone  (reference  [w])  that 

p^’ ^(y).  w) 

p (y)  * / y y (do») 

J(n  .(y),  U>,  t) 


and 


pt(y) 


/ 


•1 

p(T,wt(y,’w) 
J(^^(y),w,t) 


b,(/oT^^slV.(y),'a)di)  y(du,)’ 
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so  that  D_(^,y,  t)  = E .(b'(/  J ^(X  , s)do)]  p (y)  coincides  with  the  definition  of  D 
x y x use  x 

in  Stone  (reference  {w}),  and  the  conditions  in  that  reference  and  in  Persiheiroo 

.1 

{ (reference  [x])  are  a special  case  of  conditions  (Vl-6)  provided  one  makes  the  obvious 

changes  for  the  bound  B on  effort  density  which  is  allowed  as  a constraint  by  Stone 
reference  fvj. 

Continuous  time-discrete  space  (necessity).  The  necessity  part  of  Theorem  1 
also  holds  for  a continuous  time  and  discrete  space  motion  process.  Again  the  proof 
Is  not  a simple  extension  of  the  one  given  for  Theorem  1.  but  one  can  parallel  the  proof 
of  Theorem  5. 2 in  reference  [w]  to  obtain  the  result.  However,  in  the  case  of  discrete  space, 
the  assumption  of  concavity  for  b is  required  in  order  to  guarantee  the  necessity  of 
conditions  (VI-6).  The  concavity  is  needed  on  page  464  of  reference  [u]  where  one 
Invokes  a Lagrange  multiplier  result  to  guarantee  the  existence  of  X (t)  to  satisfy  <5.  9) 

, at  the  bottom  of  that  page.  In  the  case  of  a discrete  search  Bpace  one  must  Invoke  a 

| result  such  as  Corollary  B.  1. 2 of  reference  [h]  which  requires  the  concavity  of  the 

I detection  function  b. 

* Unified  statement  of  results.  Most  of  the  above  results  can  be  consolidated  Into 
a single  theorem  statement  provided  we  make  the  appropriate  identifications  for  D^, 

* 

Pt,  and  (0,  T).  Specifically,  D^,  is  given  by  (VI-4)  when  time  is  discrete  and  by  (VI-16) 
when  time  is  continuous;  p{  is  the  probability  density  funotion  for  when  Y is  Euolidean 
n space,  and  p (y)  ■ Pr{X  « y}  when  Y is  a discrete  space.  For  continuous  time 

I l v 

I 

F,  F , ♦(m),  and  are  defined  as  in  the  first  seotlon  but  with  Integrals  replacing  | 

| summation.  In  discrete  time  we  understand  (0,  TJ  ■ {0,  1,  ....  T}  while  in  continuous 

< I 

time  (0 , T ) has  the  usual  meaning.  1 


THEOREM  1'.  Suppose  b is  concave  and  that  it  has  a bounded  nonnegative 


derivative  b' . Assume  that  the  sample  paths  of  { X ; 0 < t < T}  are  Borel  measurable 

l 

and  that  D is  well  defined  for  (y,t)  such  that  p (y)  > 0.  Then  i[> * is  T -optimal  within 

X V 

^ (m)  if  and  only  if  there  exists  A :[0,  Tl-fO,”)  such  that 

d (**, y.t)  = A (t)  if  4'*(y,t)  > 0, 

< X (t)  if  t)  = 0 for  a.  e.  (y,  t)  e Y x [0,  T]. 

In  the  case  where  the  search  space  is  $ , the  necessity  of  conditions  (VI-18) 

n 

remains  true  when  the  concavity  assumption  on  b is  dropped.  In  discrete  time,  the 
sample  paths  will  always  be  Borel  measurable,  and  will  be  well  defined  for  (y,t) 
such  that  pt(y)  > 0. 

The  Special  Case  of  an  Exponential  Detection  Function 

When  the  detection  function  is  exponential,  we  may  prove  the  necessity  of  the 
conditions  in  (VI-6)  in  an  elementary  manner  which  requires  the  use  of  only  the 
necessary  conditions  for  an  optimal  stationary  target  search.  The  use  of  the  exponen- 
tial detection  function  also  allows  us  to  consider  the  possibility  that  the  detection 
capability  of  the  search  sensor  varies  over  the  search  space.  Specifically  we  shall 
assume  that  for  each  point  yt  Y,  there  is  a number  W(y)  which  characterizes  the 
detection  performance  of  the  sensor  in  the  neighborhood  of  y in  the  sense  that  if  the 
target,  is  located  at  y and  z effort  density  is  placed  there,  then  1 - exp(-W(y)z)  is  the 

probability  of  detecting  the  target.  Classically  W(y)  is  called  the  sweep  width  of  the 

* * 

. ** 

sensor  when  operating  in  the  neighborhood  of  y (where  1/z  has  dimensions  of  distance). 
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For  this  case 


PTW  - 1 - E 


b (■ 


2 W(X  ) ^(X 
8*0  8 8 


■•»)] 


for  4> « F , 


and  Dt>  the  kernel  of  the  linear  functional  • ],  becomes 


DT(H>,y,t)  * E 


/ T \ ‘ 

exp  y-  2 W(Xs)  *(Xg,s)J  Pt(y)W(y) 


for  ijie  F,  ye  Y,  t * 0,  1,  ....  T. 


THEOREM  2.  Suddosc  the  detection  function  is  exDonential  and  W is  bounded. 


Then  ip*  is  T -optimal  within  »(m),  If  and  only  if  there  exists  (X  (0) \(T))e 

such  that  for  t * 0,  . . , T and  for  a.  e.  ye  Y, 


Jexp^-  £ W(Xfl)  ^*(X8,s)^  j Pt(y)  W(y)e"W(y)^  * A (t)  If  ^*(y,t)  > 0 

‘ < \<t)  if  **<y.t)  = 0. 


Proof.  Since  Py  as  defined  in  (VI- 19)  is  a concave  functional,  the  sufficiency 
proof  for  Theorem  1 applies  here  also. 

To  prove  the  necessity  part  of  the  theorem,  let 


' Eyt  [ 


exp  - 2 W(X 

' art  1 


g)  ^*(Xgl  e)^  j pt(y)  for  ye  Y,  t - 0,  1 T, 


K(t)  - /ygt(y)dy  for  t - 0,  1 T. 


Suppose  that  for  some  t,  equation  (VI-20)  fails  to  hold  a set  of  positive  measure 


in  Y.  Dividing  gt  by  K(t)  to  obtain  a probability  density  gt,  we  observe  that  **(•  ,t) 
fails  to  satisfy  the  necessary  conditions  of  Corollary  2. 1. 7 of  reference  [h]  for 
4>  *(. , t)  to  maximize  probability  of  detection  for  cost  m(t)  for  a stationary  target  with 
probability  density  gfc.  Thus  4>*(- , t)  is  not  optimal  for  cost  m(t)  for  this  stationary 
target  problem. 

For  nonnegative  Borel  measurable  functions  f defined  on  Y,  let  Q[f]  be  the 
probability  of  detecting  a stationary  target  with  location  distribution  g(  and  effort 
allocation  f,  i.e. , 

Q(fi  * /y  gt(Y)  (l-e'W(y)f(y))dy. 

Since  , t)  is  not  optimal  for  cost  m(t),  we  may  find  an  f * > 0 such  that  f^t*(y)dy  - m(t) 
and  Q(f*J  > , t)J.  Observe  that 

1 - PT(V'*J  - /y  gt(y)  exp(-W(y)^*(y,t))dy 
- 1 -K<t)QHT(-,t)J 
> 1 - K(t)  Q[f*J. 

Thus,  by  taking 

( for  s it 

\ f*(y)  for  a » t, 

we  have  m)  and  which  contradiots  the  assumption  that  ip*  is 

T-optlmal  within  + (m).  Thus  equation  (VI-20)  must  bold  and  Theorem  2 is  proved. 


I 


I 


1 

I 

j 

From  the  definition  of  gfc(y)  and  K(t),  one  can  see  that  gt  = gt/K(t)  Is  eimply  j 

the  probability  density  for  the  target's  location  at  time  t given  that  it  was  undetected 

by  the  search  applied  at  all  times  other  than  t.  Let  y(t)  = X(t)/K(t)  for  t = 0,  1 T.  \ 

Then  conditions  (VI-20>  become  1 

j 

J 

gt(y)  W{y)e“W(y)^’(y,t)  = y(t)  if  <P*(y,t)  > 0, 

<y(t)  if  = 0,  . 

which  are  precisely  the  necessary  and  sufficient  conditions  for  ip*(-  ,t)  to  be  an 
optimal  allocation  of  effort  for  a stationary  target  with  probability  density  g^.  Thus 
the  optimal  moving  target  plan  can  be  characterized  in  terms  of  optimal  stationary 
target  plans.  That  is  at  each  time  tc0,  1,  ....  T,  the  optimal  plan  ip*  allocates 

•] 

the  effort  available  at  time  t so  that  **(•  ,t)  maximizes  the  probability  of  detecting 
stationary  target  with  probability  density  g . 


i 

i 
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